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FOREWORD 


Tlil«  rejpc rt  wa»  prepared  by  the  University  of  Dayton  Research 
Institute,  Dayton,  Ohio,  under  Air  Force  Coutract  AF  33(615)-3389.  The 
work  deecribed  herein  wae  accomplished  under  Project  No.  7381  "Materials 
Application,  "  and  Task  No.  738103,  "Materials  Information  Development, 
Collection  and  Proceeding.  "  The  effort  waa  administered  under  the  direction 
of  the  Materials  Information  Branch,  Materials  Applications  Division,  Air 
Force  Materials  laboratory  with  H.  B.  Thompson,  MAAM,  as  project 
monitor. 

This  is  a  summary  technical  report  aivd  covers  the  work  accomplish¬ 
ed  from  1  December  1966  through  30  November  1967. 

The  author  acknowledges  the  efforts  and  contributions  of  the  Project 
Supervisor,  Fdward  A.  Janning,  his  co-worker*  Eugene  R.  Egan,  Ralph  B. 
Smith,  and  Linda  C.  Haley,  and  the  student  indexers,  Patricia  A.  Thomas 
and  Edward  J.  Fruin.  Dr.  Ivan  Coldfarb  from  the  Polymer  Branch,  AFML, 
served  as  a  consultant  on  search  profile  preparation  and  Edward  L.  Horne 
of  the  Materials  Information  Branch,  AFML,  coordinated  the  CAS-SDI 
program. 

Thie  report  was  submitted  by  the  author  November  1967. 

This  technical  report  has  been  reviewed  and  is  approved. 


D.  A.  SHINN 

Chief,  Materials  Information  Branch 
Materials  Applications  Division 
Air  Force  Materials  Laboratory 
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ABSTRACT 


Two  graduate  students  in  client! airy  were  trained  in  indexing  of  tech¬ 
nical  documents  through  an  established  training  program.  Certain  modifica¬ 
tions  were  made  which  tended  to  improve  the  training  as  evidenced  by  im¬ 
proved  performance,  A  sivdy  of  infra-  and  interindexer  consistency  was 
made  by  subjecting  a  group  of  technical  documents  to  independent  indexing 
by  various  individuals  and  by  twn  of  the  same  individuals  one  year  later. 

The  concept  of  essential  index  terms  was  introduced.  Statistical  analysis 
showed  that  Intra-  and  interindexer  correlation  for  experienced  indexers  of 
essential  term#  i*  significant  with  n  probability  (p)  ;?  0.999. 

An  evaluation  of  the  indexer  trainees  showed  that  essential  terms  was 
a  better  performance  criterion  than  total  terms  and  that  high  interindexer  con¬ 
sistency  was  achieved.  Search  requests  declined  from  the  preceding  year, 
especially  from  the  AFML.  Possible  reasons  for  the  decline  are  presented. 

A  Selective  Dissemination  of  Information  (£>DI)  Program  with  Chemical 
Abstracts  was  initiated.  The  development  of  a  vocabulary  and  thesaurus  for 
Information  science  related  documents  has  started.  A  free  indexing  technique 
is  being  used. 
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SECTION  I 


INTRODUCTION 


The  Information  Systems  Section  of  the  University  of  Dayton  Research 
Institute,  under  contract  to  the  Air  Force  Materials  Laboratory  (AFML),  has 
established  and  presently  maintains  a  document  retrieval  system  in  support 
of  the  Aerospace  Materials  Information  Center  (AMIC).  The  AFML,  through 
its  Materials  Information  Branch,  has  established  a  network  of  seven  specia¬ 
lized  information  and  data  centers  throughout  the  country  which  constitute  the 
Air  Force  Materials  Information  Centers  (AFMIC).  The  purpose  of  AFMIC 
is  to  acquire,  process,  store,  retrieve,  and  disseminate  materials  data  and 
information  to  members  of  the  scientific  and  technical  community  engaged  in 
research  and  development  for  the  Department  of  Defense  (DOD).  The  Air 
Force  encourages  the  use  of  these  centers  which  provide  their  services  to 
Government  agencies.  Government  contractors,  suppliers  and  others  who 
are  in  a  position  to  contribute  to  the  defense  effort. 

The  document  retrieval  system  operated  by  the  University  of  Dayton 
contains  approximately  31,  000  documents  related  to  materials  research  and 
development  with  new  accessions  being  added  continually.  The  establishment, 
modification,  and  operation  of  the  document  retrieval  system  are  described 
in  References  6,  7,  8,  and  10.  A  recapitulation  of  these  reports  is  presented 
in  Appendix  L 

This  report  covers  the  work  accomplished  from  December  1966  to 
December  1967.  The  indexer  training  program  developed  and  described  in 
Reference  10  was  used  in  indexing  training  of  two  graduate  students  in  chemis¬ 
try  and  a  high  school  teacher  in  physics.  Although  the  same  basic  program 
was  followed,  certain  modifications  were  included  which  improved  the  effec¬ 
tiveness  of  the  training.  A  study  of  the  criteria  for  evaluating  new  indexers 
was  made  and  some  interesting  results  of  intra-  and  interindexer  consistency 
were  obtained  by  statistical  analysis.  A  pilot  Selective  Dissemination  of  In¬ 
formation  (SDI)  program  was  inaugurated  with  Chemical  Abstracts  Service 
(CAS).  The  CAS  services  subscribed  to  were  the  SDI  experiment,  FOST-P, 
and  POST-J  (See  Section  IV).  Various  changes  in  the  operations  of  the  docu¬ 
ment  retrieval  system  were  made,  and  documents  related  to  information 
science  are  being  indexed  for  eventual  inclusion  in  the  system.  The  termi¬ 
nology  of  information  science  presents  special  problems  for  indexing  and 
thesaurus  development  which  will  be  considered  before  a  viable  retrieval  sys¬ 
tem  for  these  documents  can  be  established.  An  exciting  new  concept  called 
computer  assisted  indexing  has  been  evolved  for  processing  of  documents. 

This  study  has  just  commenced.  The  acquisition  of  ar  IBM  360  computer  at 
the  University  of  Dayton  in  conjunction  with  McCall  Corporation,  and  the 
scheduled  acquisition  of  remote  terminal  access  by  the  Information  Systems 
Section  will  provide  the  hardware  necessary  for  this  study. 
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SECTION  U 

USE  OE  GRADUATE  STUDENTS  AS  INDEXERS 


Lm tit  y *»f,  encouraging  results  »rr«  obtained  with  *  rhrmlitfy  lludnit, 
who  had  Joat  graduated,  and  another  student,  who  had  Just  completed  her  Junior 
year  in  chemletry,  as  indexers.  This  experience  showed  the  feasibility  of  us- 
ing  technically  trained  atudenta  for  indexing  materials  documents  and  suggest¬ 
ed  the  employment  of  additional  student  pcraonual.  During  the  Bummer  uf 
1967,  two  graduate  student*  in  chemistry  were  obtained  on  graduate  atslat&nl- 
ahips  In  cooperation  with  the  Department  of  Chemistry.  In  the  graduate  as- 
sistantahip  program,  the  student*  work  full  time  during  the  Summer  and  hxU- 
time  during  the  school  year  while  attending  classes.  It  is  a  two-year  program, 
thus  providing  the  services  of  the  students  for  an  extended  period  of  time  after 
they  have  been  trained. 

The  two  students  started  to  work  early  in  July  1967  and  their  training 
followed  very  closely  that  described  in  detail  in  Reference  10.  About  the 
same  '.h..w,  a  high  school  physics  teacher  was  employed  for  full  time  summer 
employment  and  part-time  work  during  tne  »G»ooi  year.  Th«  ihret  person* 
had  the  opportunity  to  proceed  through  the  training  program  at  the  same  time. 

There  were  certain  aspects  of  the  original  training  program  which 
were  modified  ae  indicated  by  the  progress  of  the  indexer  trainees.  The  ex¬ 
tracting  function  was  de-emphasixed  to  some  extent,  and  direct  indexing 
from  the  document  was  started  earlier.  The  extracting  of  documents,  that 
is,  the  writing  out  of  concepts  in  word,  phrase,  or  sentence  form,  tended  to 
regiment  one's  thinking  to  consideration  of  the  document  as  a  whole.  Further, 
extracting  tended  to  require  excessive  time,  and  the  procedure  was  not  at  all 
commensurate  with  the  benefit  derived.  Document  analysis,  however,  was 
effective  In  learning  the  indexing  function.  As  will  be  shown  In  the  next  sec¬ 
tion,  the  average  indexing  time  required  for  the  series  of  documents  used  In 
experimental  testing  was  remarkably  less  for  this  year's  trainees.  The  total 
formal  training  peried  was  reducad  by  25%,  from  four  weeks  in  1966  to  three 
weeks  In  1967,  Comparisons  of  the  students  from  both  years  are  presented 
in  Section  ITL 

It  can  be  stated  that  the  results  of  the  Indexer  training  program  from 
1966  were  very  good,  and  the  results  from  1967  were  even  better.  There 
are  several  factor*  which  may  account  for  this.  The  good  results  from  both 
years  attributed  to  considerable  Interaction  between  the  trainees  and  the 
supervisors.  The  trainees  received  rapid  feedback  at  first  by  independ 
indexing  of  the  same  documents  by  the  eupervisors  and  trainees  and  later  by 
review  Indexing  of  the  trainees'  index  carde.  The  succees  of  the  program  is 
also  due  to  some  extent  to  the  exposure  of  the  students  to  the  entire  operation 
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of  the  document  retrieval  system.  In  this  way,  they  see  the  importance  of 
the  original  indexing  to  the  other  functions,  especially  in  the  retrieval  of 
documents.  Also  the  students  are  periodically  apprised  of  other  Information 
Systems  Section  projects  and  are  exposed  to  other  information  systems. 

The  indexers  are  performing  professional  work  of  basic  importance 
to  the  continued  operations  of  the  document  retrieval  system,  and  they  are 
treated  as  professionals.  The  indexers  are  encouraged  to  contribute  ideas 
for  improving  operation,  and  numerous  worthwhile  suggestions  have  been 
received. 

The  rate  of  advancement  of  the  indexer  trainees  was  more  rapid  this 
year  than  last  year.  One  reason  for  this  was  that  the  administration  of  the 
training  program  was  better,  since  experience  had  been  gained  from  applying 
the  program  the  preceding  year.  The  de- emphasizing  of  the  extracting  pro¬ 
cedure  was  also  helpful.  Concomitant  with  these  factors  the  indexers  were 
more  experienced  and  mature,  and  hence  their  ability  to  grasp  the  material 
more  quickly  was  enhanced.  Since  the  graduate  assistants  are  committed  to 
pursuing  their  graduate  course  of  study  at  the  University  of  Dayton  for  two 
years,  they  may  be  more  highly  motivated  than  the  students  who  were  em¬ 
ployed  only  for  the  summer. 

In  conclusion,  it  may  be  stated  that  the  indexer  training  program  de¬ 
veloped  at  the  University  of  Dayton  is  highly  effective  in  producing  competent 
indexers.  Both  undergraduate  and  graduate  students  have  been  trained,  with 
somewhat  better  results  being  obtained  with  graduate  students.  It  cannot  be 
stated  categorically,  however,  that  graduate  students  necessarily  become 
better  indexers,  since  other  factors  were  operating  which  may  as  well  ac¬ 
count  for  this  result. 
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SECTION  III 


A  STUDY  OK  INTHA-  AND  INTER  INDEXER  CONSISTENCY  AND  INDEXING 

KVAI.ITATIuN 


As  reported  In  Referenc  e  10,  an  evaluation  procedure  ha*  been  de¬ 
veloped  o  evaluate  the  progress  <->f  indexer  trainees.  In  essence,  the  tech  ¬ 
nique  compared  the  indexing  terminology  used  by  the  student*  with  the  total 
terms  used  by  the  supervisor  in  indexing  and  also  with  the  essential  term* 
used  by  the  supervisor  in  indexing.  The  time  required  for  indexing  was  also 
compared.  It  wa*  suggested  that  a  fairer,  mure  objective  evaluation  would 
be  made  by  using  the  essential  terms  a#  a  basis  fur  comparison  rather  than 
by  using  the  total  terms  used  by  the  supervisor.  Implicit  in  using  the  super¬ 
visor's  indexing  as  a  reference  standard  is  an  assumed  reliability  or  con¬ 
sistency  of  the  supervisor  himself  regarding  his  indexing. 

Tli*  "t(  of  essential  terms  suggests  that  these  terms  have  certain 
characteristics  which  distinguish  them  along  a  continuous  subjective  param¬ 
eter  of  "essentiality"  in  that  index  terms  can  be  judged  as  being  essential  or 
nonessentiai.  One  could  state  that  if  an  index  term  is  listed,  it  must  be 
essential  or  it  would  not  be  listed.  However,  the  philosophy  of  indexing  em¬ 
ployed  in  the  Information  Systems  Section  is  that  while  there  are  essential 
terms  which  must  be  listed,  there  aiv  uther  terms  which  may  well  be  helpfuL 
in  indexing  and  retrieval  but  which  do  not  have  the  character  of  being  essen¬ 
tial.  The  judgment  of  essential  terms  is  tempered  by  experience  with  search 
requests.  The  frequency  with  which  terms  are  used  in  formulating  search 
strategies  has  some  influence  on  the  judgment  of  what  constitutes  essential 
terms.  Since  the  AMIC  document  retrieval  system  is  oriented  toward  ma¬ 
terials  information,  materials  terms  are  usually  essential  terms,  and  this 
is  reflected  in  the  user  requests.  For  a  system  fulfilling  a  different  mission, 
e.  g.  ,  a  system  containing  human  factors  information,  materials  terms  prob¬ 
ably  would  not  be  considered  essential  terms. 

Assuming  "ideal"  indexing  la  not  always  possible,  it  is  considered 
better  to  "overindex"  a  document,  risking  a  false  retrieval,  than  to  "under¬ 
index"  the  document,  risking  r.onr etr ieval  of  a  pertinent  item.  If  the  docu¬ 
ment  is  retrieved  and  presented,  the  requestor  at  least  ha*  the  option  of  ac¬ 
cepting  or  rejecting  it!  if  it  i*  not  retrieved,  this  selection  option  cannot  be 
exercised.  Generally,  terms  from  the  title,  terms  from  the  abstract,  and 
materials  terms  are  considered  essential,  but  there  are  many  exceptions  to 
this  generalization  depending  or  the  document,  the  subject  matter,  etc. 
Essential  terms  should  be  less  subject  to  variation  than  the  helpful,  but  not 
essential,  terms. 
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Studies  of  intra-  and  interindexer  consistency  were  undertaken  to 
determine  the  validity  of  using  an  experienced  indexer's  indexing  as  a  refer¬ 
ence  standard  and  to  assess  the  quality  of  indexing  among  various  indexers. 
One  might  consider  the  following  question:  if  a  document  were  given  to  an 
indexer  for  indexing,  what  degree  of  correlation  of  terms  would  result  if  the 
same  document  were  indexed  by  the  same  indexer  at  a  different  time?  An¬ 
other  question  would  be  as  follows:  What  correlation  of  terms  would  result 
from  two  independent  indexers  indexing  the  same  document?  If  it  can  be 
shown  that  statistically  significant  correlation  exists,  evidence  would  be  pro¬ 
vided  for  assessing  intra-  and  interindexer  reliability  or  consistency.  It 
was  to  the  answering  of  these  questions  that  the  following  studies  were 
directed. 

In  1966,  35  documents  were  selected  which  dealt  primarily  with 
chemistry.  These  documents  were  independently  indexed  by  both  trainees 
and  the  supervisor,  an  experienced  indexer,  about  one  week  after  the  trainees 
had  completed  their  training  program.  Several  months  later  the  experiment 
was  repeated  with  another  series  of  documents.  Comparisons  were  made  of 
essential  terms,  total  terms,  and  time  required  for  indexing.  The  student 
trainees  were  instructed  to  index  the  document  as  well  as  they  could:  they 
were  unaware  cf  the  essential  terms  until  the  completion  of  the  first  experi¬ 
ment.  The  students'  indexing  was  simply  compared  with  the  supervisor's 
judgment  of  essential  and  total  terms  as  a  reference  standard.  The  results 
were  reported  in  tabular  form  in  Reference  10. 

In  1967,  the  first  set  of  35  documents  was  recalled  from  the  AFML 
Library  and  indexing  was  performed  by  the  two  new  trainees  and  by  two 
experienced  indexers,  one  of  whom  was  the  supervisor  in  1966.  Both  ex¬ 
perienced  indexers  were  instructed  to  designate  the  essential  terms  for  each 
indexed  document.  Although  the  same  documents  were  re-indexed,  there 
was  little  noticeable  learning  effect,  since  an  entire  year  had  elapsed  with 
many  documents  being  indexed  during  the  year  by  both  experienced  indexers. 
One  Indexer,  however,  indicated  that  the  prior  indexing  had  made  the  docu¬ 
ments  somewhat  familiar  to  him  which  probably  caused  a  reduction  in  index¬ 
ing  time  over  what  would  have  been  expected  through  additional  indexing 
experience. 

By  comparing  the  indexing  performed  by  the  same  indexer  of  the  same 
document,  a  statistical  correlation  coefficient  can  be  obtained  for  essential 
terms  which  serves  as  a  measure  of  intraindexer  reliability  or  consistency. 
Likewise  the  common  essential  terms  can  be  obtained  between  two  experi- 
enced  indexers  as  a  measure  of  inter  indexer  consistency. 

The  statistical  correlation  techniques  used  are  the  phi  coefficient  (<$) 
and  chi  square  (X^).  The  phi  coefficient  is  particularly  suitable  for  deter¬ 
mining  the  degree  of  association  between  two  dichotomous  variables,  that  is, 
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variable*  which  can  be  divided  into  two  portion*.  The  value  of  the  <&  coeffici¬ 
ent  is  an  indication  cf  the  degree  to  which  the  two  dichotomous  variable*  are 
associated.  The  Q5  coefficient  can  also  be  applied  to  continuous  variables 
which  are  arbitrarily  dichotomised.*  For  example,  one  may  wish  to  deter¬ 
mine  the  correlation.  If  any.  o/hirh  exists  between  family  income  and  owner¬ 
ship  of  common  etoc.ke.  The  ownership  of  common  stock*  is  already  dicho¬ 
tomous  since  a  family  either  owne  stock  or  doe*  not  own  stock.  Tins  family 
Income  can  be  dichotomized  by  arbitrarily  setting  a  level  of  $10,000  and 
considering  family  income  greater  than  $10,  000  and  less  than  $10,  000.  There 
are  fou^  possible  situation*  which  could  occur:  1)  families  with  income  greats 
er  than  $10,  000  who  own  stock;  2)  families  with  income  less  than  $10,  000  who 
own  stock;  3)  families  with  income  greater  than  $10,  000  who  do  not  own  stock; 
4)  families  with  income  less  than  $10,000  who  do  not  own  stock,  If  one  were 
to  survey  100  families  regarding  income  and  stock  ownership,  the  families 
could  be  categorized  into  the  four  categories.  A  Z  x  2  table  could  be  con¬ 
structed  as  follows: 

INCOME 


STOCK 

OWNERSHIP 


This  table  shows  that  of  the  50  families  of  income  greater  thau  $10,  000,  40 
own  stock,  10  do  not  own  stock.  Of  the  families  of  income  less  than  $10,  000, 
15  own  stock,  35  do  not  own  stock.  The  table  indicates  that  there  may  be  a 
correlation  between  family  income  and  stock  ownership.  The  <3  coefficient  is 
a  quantitative  expression  of  the  degree  of  correlation,  labeling  the  cells  of 
the  paradigm  A,  B,  C  and  D,  the  d  coefficient  is  calculated  as  follows: 


BC  -  .AD 


a 


V  (A+B)  (C-iD)  (A+C)  (B+D) 


(40)  (35)  -  (15)  (10) 


(Reference  4) 


1250 


3,  503 


Vl55)  (45)  (50)  (50) 


(50)  (49.  7) 


The  maximum  d  coefficient  would  be  1.  0.  This  <$  coefficient  would  be  ob¬ 
tained  if  all  families  have  income  greater  than  S10,  000  owned  stock  and  none 
of  the  families  with  Income  less  than  $10,000  owned  rtock.  The  paradigm 
would  appe-i  as  follows; 


€  $10, 000 

>  $10.  000 

Own  Stock 

(a) 

15 

(B) 

40 

55 

Do  not 

<c> 

(D) 

Own  Stock 

35 

10 

45 

i 

50 

50 

100 
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Chi  square  is  a  test  of  statistical  significance.  A  significance  test  is 
a  mathematical  procedure  applied  to  empirical  data  for  deciding,  on  grounds 
of  probability,  whether  or  not  a  suitably  formulated  hypothesis  is  tenable. 

An  observational  result  is  said  to  be  significant,  and  the  hypothesis  in  ques¬ 
tion  is  rejected,  when  the  obtained  result  belongs  to  an  objectively  specified 
unfavorable  class  having  a  fitted,  small  probability  of  occurrence  in  random 
samples  from  the  hypothesized  population.  In  the  case  of  testing  the  stock 
ownership  -  family  income  correlation,  the  following  hypothesis  is  made: 
stock  ownership  is  not  dependent  on  family  income.  The  alternate  hypothesis 
is:  stock  ownership  is  dependent  on  family  income.  The  obtained  result  must 
belong  to  the  objectively  specified  unfavorable  class  having  a  fixed  small  pro¬ 
bability  of  occurrence  (level  of  significance)  in  random  samples  from  the 
hypothesized  population.  If  such  a  result  is  obtained,  the  hypothesis  is  re¬ 
jected  and  the  alternative  hypothesis  is  accepted,  i.  e.  stock  ownership  is 
dependent  on  family  income.  Usually  the  probability  at  which  the  results  are 
significant  is  indicated  with  the  results. 

Chi  square  can  be  used  to  determine  "goodness  of  fit"  of  data.  In  the 
example  given  above,  a  d  coefficient  of  0.503  was  obtained.  The  maximum 
possible  GJ  coefficient  was  1.0.  One  might  ask  whether  the  observed  results 
represent  a  statistically  significant  "goodness  of  fit"  to  the  GS  coefficient  of 
1.0.  Chi  square  (X^)  is  related  to  the  C 6  coefficient  by  the  equation  X^  = 
where  N  is  the  total  number  of  individual  items,  in  this  case  the  number  of 
families. 

Calculating  X^:  X2  =  NGJ2  (Reference  4) 

X2  =  100  (.  503)2 
X2  =  25.  3 
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.  . S’  .  .  . 

Referring  this  value  to  a  statistical  tame  ot  a~  it  can  oe  aetermtnee  that  tnis 

result  is  significant  at  a  probability  (p}  >  0,999.  Therefore  there  is  statisti¬ 
cal  evidence  to  support  the  hypothesis  that  stock  ownership  is  correlated  with 
family  inccme.  Referring  to  Figure  1,  it  can  be  seen  that  the  X2  value  of 
28,  3  for  N=1Q0  falls  on  the  curve  above  the  line  indicating  p=0.  999. 

The  <$  coefficient  and  chi  square  can  be  employed  to  determine  the 
correlation  between  essential  index  terms,  Chi  square  has  been  used  previ¬ 
ously  to  determine  the  degree  of  term  association  i.  e.  the  extent  to  which 
terms  appear  together  in  indexing,  ^  Once  Judgments  have  been  made  of 
essential  terms  contained  in  a  list  of  terms,  the  remaining  terms  are  neces¬ 
sarily  nonessential  terms.  Therefore,  the  index  terms  constitute  a  dicho¬ 
tomous  variable.  However,  prior  to  the  decision  of  whether  a  given  term  is 
essential  or  not,  the  decision  itself  is  a  subjective  matter  dependent  on  the 
experienced  indexer  who  must  apply  his  judgment,  talcing  into  account  the 
essentiality  of  the  term  in  the  context  of  the  report  and  in  relation  to  the  par¬ 
ticular  retrieval  system.  In  the  example  of  income  and  stock  ownership, 
the  ownership  of  stock  is  not  at  all  subjective  nor  is  the  amount  of  income. 

The  essential  and  noneasential  terms  are  compared  by  groups  and 
the  degree  of  correlation  is  determined  between  groups.  For  Intraindexer 
consistency,  the  first  group  of  terms  is  taken  from  one  year's  indexing  and 
compared  with  the  second  group  of  terms  from  the  other  year's  indexing,  all 
taken  from  the  same  set  of  documents.  The  results  are  given  in  Table  I  and 
in  Figure  1.  The  calculations  arc  presented  in  Appendix  IL  The  results 
show  statistically  significant  correlation  for  essential  terms  regarding  intra- 
indexer  and  interindexer  consistency. 

2 

Referring  to  Figure  1,  it  can  be  seen  that  the  values  for  X  lie  well 
above  the  p=.999  level  of  significance.  The  value  of  which  represents 
statistical  significance  at  p=0.  999  is  10.83.  The  curves  representing  X2 
as  a  function  of  the  d  coefficient  show  that  the  lumber  of  terms  included  in 
the  study  has  a  considerable  effect  on  the  relationship.  The  degree  of  corre¬ 
lation  (d  coefficient)  for  a  large  number  of  terms  can  be  relatively  small  and 
still  be  statistically  significant.  On  the  other  hand,  with  a  small  number  of 
terms,  a  fairly  high  degree  of  correlation  is  required  for  statistical  signifi¬ 
cance.  The  curves  also  demonstrate  that  as  the  degree  of  correlation  in¬ 
creases,  i.e.,  as  the  d  coefficient  increases,  the  corresponding  X2  increases 
at  a  much  faster  rate.  Thus  a  small  incremental  improvement  in  the  degree 
of  correlation  gives  a  large  incremental  increase  in  the  level  of  significance. 
These  results  tend  to  confirm  that  an  experienced  indexer'B  indexing  with 
judgments  of  essential  terms  car.  serve  as  a  reference  standard  for  the  evalu¬ 
ation  of  indexer  trainees.  The  comparisons  made  in  the  remainder  of  this 
section  will  be  based  on  the  indexing  of  FLiS  in  1966  as  a  reference. 


8 


An  interesting  observation  wu  made  regarding  Intraindexer  consist¬ 
ency.  With  consideration  of  terms  Judged  essential  in  1966  and  terms  Judged 
essential  in  1967,  7  3%  of  the  terms  judged  essential  in  1967  matched  the  essen¬ 
tial  terms  of  1966.  However,  considering  the  entire  list  of  terms  from  1967 
without  regard  to  their  judged  essentiality,  86%  of  the  essential  terms  of  1966 
were  included  in  the  indexing  of  1967.  If  the  procedure  is  inverted,  i.  e. ,  1967 
is  taken  as  the  reference  standard,  the  corresponding  figures  are  68%  and  92%. 
The  raauits  indicate  that  there  is  a  "core"  of  csscitial  term*  (essential  terms 
of  1966  plus  essential  terms  of  1967,  minus  duplicated  terms),  which  was  used 
in  Indexing  both  years,  although  there  is  some  discrepancy  regarding  the  judg¬ 
ment  ox'  essentiality  of  terms  within  the  core  between  the  two  years.  The  situ¬ 
ation  is  illustrated  in  Table  II,  It  should  be  noted  that  intraindexer  consistency 
would  be  expected  Intuitively  to  be  greater  than  interindexer  consistency. 

Figure  2  and  Table  IQ  show  the  comparisons  of  all  indexers  involved 
in  the  study  from  both  ye  irs  for  the  period  immediately  following  the  training 
period.  The  improvement  over  the  Summer  period  made  by  the  indexers  of 
1966  is  indicated  by  dotted  line  extensions  in  Figure  2.  The  improvement 
was  determined  by  performing  an  experiment  with  another  set  of  35  documents 
at  the  end  of  the  summer.  Appendix  III  presents  the  indexing  of  all  indexers 
involved  for  a  typical  document. 

Certain  trends  are  evident  in  the  data  obtained.  In  the  first  place,  the 
experienced  indexer  of  both  years  uued  more  terms  for  indexing  in  1967  than 
in  1966.  Slightly  more  terms  (about  7%  more)  were  indicated  as  being  essen¬ 
tial,  although  the  percentage  of  terms  considered  essential  based  on  the  total 
index  terms  decreased  from  48%  to  43%.  This  finding  further  verifies  the 
stability  or  reliability  of  essential  terms  and  indicates  that  more  noneisen- 
tial  or  "helpful  but  not  absolutely  essential"  terms  were  used  in  indexing  in 
1967.  The  time  required  for  indexing  was  reduced  considerably  even  though 
more  index  terms  were  used.  It  was  indicated  earlier  that  the  indexer  felt 
some  familiarity  with  the  material  even  though  a  year  had  passed  between  in¬ 
dexing  and  reindexing. 

In  comparing  the  performance  of  the  two  experienced  indexers  in  1967, 
it  can  be  seen  that  one  of  them  (LCH)  used  more  index  terms  and  considered 
more  terms  essential  than  the  other  (FL3),  The  sampling  and  comparison  of 
Individual  index  cards  show  that  the  terms  judged  essential  by  LCH  are  nearly 
identical  to  the  "core"  of  essential  terms  derived  from  independent  Judgments 
of  essential  terms  by  FLS.  This  fact  enhances  even  further  the  validity  of 
essential  terms  as  a  reference  standard.  The  time  for  indexing  the  docu¬ 
ments  was  nearly  Identical  for  the  two  experienced  Indexers  in  1967. 

The  graduate  students  were  decidedly  better  in  their  performance 
than  the  students  in  1966,  Their  indexing  time  was  remarkably  less  and 
their  index  terms  were  somewhat  more  in  agreement  with  the  experienced 
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TAB1E  II.  COMPARISON  OF  INDEXING  OF  DOCUMENT  29748  FOR  THE  SAME  INDEXER  IN  1966  AND  1967 


<r 


Ei“5§ 


oc 

•  n  Ul 


u.  f-  Q 

-j  10  < 

UJ  <71  K 
“  <3 


r~  S 
4  10 

a  <t> 


a 

< 

ac 

o 


x  n  a; 

O  W  x 
J91U 


v>  n.  a. 
_i  to  x 
U.  <71  Ul 


J _ I _ I _ I _ I _ I _ I _ I _ i _ I _ 1 _ 1 _ I _ I _ I _ I _ 1-J _ I _ L 


o 

o  o  o 

O 

O 

o  o 

o 

o 

<7! 

00  f-  <o 

in 

"9 

|<|  CvJ 

9961  -  S"U  NO 

03SVB 

SWW3JL 

JO  % 

1 1 


FIGURE  2:  COMPARISON  OF  INDEXERS  RELATIVE  TO  EXPERIENCED  INDEXERS  (fFLS  -  1966. 
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indexer  regarding  both  essential  terms  and  total  terms.  These  results  can 
be  explained  largely  on  the  availability  of  a  better  thesaurus,  the  more  ex¬ 
tensive  background  of  the  indexer  trainees,  and  the  modification  of  the  train¬ 
ing  program.  Other  possible  factors  may  also  help  to  account  for  the  im¬ 
proved  performance  as  indicated  in  Section  IL 

It  is  of  interest  to  consider  why  indexers,  and  especially,  the  experi¬ 
enced  indexers,  tended  to  use  more  total  terms  and  essential  terms  in  index¬ 
ing  in  1967  than  in  1966.  As  stated  earlier,  the  retrieval  of  a  document 
abstract  gives  the  requestor  the  opportunity  for  acceptance  or  rejection. 
Nonretrieval  of  an  item  does  not  afford  the  requestor  the  opportunity  of  see¬ 
ing, it  to  make  such  a  decision.  More  index  terms  increase  the  probability 
of  retrieval.  Another  requestor  trend  is  the  desire  for  specific  information 
which  necessitates  indexing  of  specific  concepts.  Both  of  these  requestor 
parameters  have  caused  the  philosophy  of  indexing  to  tend  somewhat  towards 
the  use  of  more  Indexing  terms  and  perhaps  a  somewhat  broader  interpreta¬ 
tion  of  what  essential  terms  are. 

Even  though  it  sounds  paradoxical,  less  time  is  required  in  Indexing 
with  more  terms.  This  phenomenon  is  clearly  indicated  in  the  results.  Prob¬ 
ably  the  explanation  is  that  decision  making  regarding  terms  is  reduced.  One 
need  not  be  quite  as  selective  in  the  index  terms  used.  If  there  is  some  doubt 
regarding  the  use  of  a  particular  term,  the  term  is  assumed  to  be  of  suffici¬ 
ent  merit  to  be  listed.  With  more  selective  Indexing,  the  decision  of  whether 
to  list  it  may  be  fairly  time  consuming,  and  these  decision  making  time  in¬ 
crements  are  additive. 

Another  reason  for  the  improved  performance  is  that  the  thesaurus 
was  revised  between  the  summer  of  1966  and  the  summer  of  1967.  The  re¬ 
vised  thesaurus  contains  a  much  larger  number  of  reference  terms  which 
indicate  how  to  express  concepts  in  active  vocabulary  terms.  The  improved 
thesaurus  undoubtedly  was  beneficial  to  the  Indexer  trainees. 
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SECTION  IV 


DOCUMENT  RETRIEVAL  SYSTEM  OPERATION  AND  AMIC  RELATED 

ACTIVITIES 


Input,  One  of  the  primary  objectives  of  the  Information  Systems 
Section  concerning  the  AFML  contract  is  the  indexing  of  newly  acquired  docu¬ 
ments  for  inclusion  into  the  document  retrieval  system  and  the  retrospective 
searching  of  the  file  in  response  to  search  requests.  Two  changes  made  in 
the  input  operation  have  improved  efficiency.  Formerly,  keypunch  cards, 
which  are  used  for  updating  the  computer  search  tape,  were  processed  by 
the  keypunch  section  of  the  Research  Institute.  During  the  summer  the  avail¬ 
ability  of  keypunch  equipment  suggested  the  use  of  our  own  clerical  personnel 
for  keypunching.  Keypunching  by  Information  Systems  Section  personnel  has 
proved  effective.  Familiarity  of  the  clerical  personnel  with  the  technical 
vocabulary  and  interest  in  their  work  has  kept  keypunch  errors  to  a  minimum. 
Further,  tighter  control  on  the  status  of  keypunching  is  maintained.  Another 
change  which  has  proved  beneficial  is  the  inclusion  of  the  Defense  Documenta¬ 
tion  Center  (DDC)  AD  number  or  the  NASA  N  number  with  each  abstract. 

This  information  is  cross-referenced  at  the  AFML  Library,  but  it  is  more 
convenient  to  have  the  AD  or  N  number  available  immediately  on  retrieval  of 
the  document  abstract.  The  document  can  then  be  ordered  if  desired  from 
DDC  or  the  NASA  information  facility. 

Microfilm  (16  mm)  reels  are  being  prepared  of  all  abstract  and  in¬ 
dex  cards.  As  indicated  in  Reference  10,  the  use  of  microfilm  provides  a 
permanent  accessible  record  and  facilitates  the  screening  of  searches.  As 
microfilming  has  proceeded,  another  advantage  has  become  evident.  The 
preparation  of  microfilm  requires  accurate  and  complete  records  of  index 
cards  and  abstracts.  Microfilming  has  revealed  a  number  of  cases  of  in¬ 
completeness  which  have  been  brought  up-to-date.  Microfilming  thus  serves 
as  an  input  control  function  as  well  as  providing  a  permanent  record. 

During  the  period  covered  by  this  report,  1  December  1966  through 
30  November  1967,  approximately  4250  documents  wore  indexed  and  process¬ 
ed  into  the  system.  The  documents  were  indexed  with  an  average  of  24  terms 
per  document  (exclusive  of  automatic  generic  postings)  with  an  average  index 
time  of  about  25  minutes.  Distribution  by  subject  category  is  shown  in 
Appendix  V.  There  are  now  approximately  31,000  documents  in  the  AMIC 
document  retrieval  system. 

Searching.  A  total  of  286  technical  requests  were  processed  by  the 
Information  Systems  Section  last  year.  An  average  of  9.0  abstracts  was 
printed  per  search  from  '.he  microfilm  records  for  Inclusion  as  part  of  the 
search  results.  User  feedback  is  very  favorable  toward  the  receipt  of  ab¬ 
stracts.  A  complete  listing  of  searches  processed  by  title,  number, 
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requestor  and  organization  is  presented  in  Appendix  IV.  It  includes  an  in¬ 
dex  of  requestors  and  corresponding  search  numbers,  and  the  number  of 
searches  requested  by  various  organizations. 

The  number  of  technical  requests  received  declined  about  26.5%  from 
last  year.  Figure  3  presents  the  total  number  of  requests  processed  by  the 
AMIC  document  retrieval  system  since  1963.  The  number  of  search  requests 
performed  for  the  AFML  and  for  all  othe  r  organizations  is  indicated  in  addi¬ 
tion  to  the  search  totals.  It  can  be  seen  readily  from  Figure  3  that  the  num¬ 
ber  of  search  requests  from  the  AFML  has  been  on  the  decline  for  the  past 
two  years.  Whereas  the  number  of  search  requests  from  other  organizations 
has  increased  with  only  a  slight  decline  this  year.  The  drop-off  in  the  num¬ 
ber  of  AFML  requests  is  a  matter  of  some  concern,  since  AFML  personnel 

should  be  prime  users  of  the  AMIC  system.  j 

Several  reasons  can  be  advanced  for  the  reduced  usage  of  the  AMIC 
system  by  the  AFML.  Every  new  project  started  in  the  AFML  is  required  ! 

to  go  through  AMIC  before  a  request  for  proposal  (RFP)  is  submitted  for 
bids.  The  project  engineer  thus  gets  a  background  of  the  state-of-the-art 
of  the  topic  he  is  pursuing.  Such  a  search  request  may  reveal  that  certain 
information  is  already  available  and  that  the  RFP  should  be  modified  to  pre¬ 
vent  exact  duplication  of  work  already  performed.  On  the  other  hand,  the 
search  may  indicate  by  lack  of  retrievals,  that  research  in  the  proposed  area 
has  not  been  done  previously,  thus  underscoring  the  need  for  proceeding  with 
the  project.  The  number  of  new  projects  initiated  in  1965  and  1966  was  for¬ 
tuitously  quite  high;  in  fact,  such  a  level  of  new  projects  could  not  be  expect¬ 
ed  to  be  maintained  each  year.  Coupled  with  this  factor  is  the  reduced  fund¬ 
ing  available  for  Air  Force  research  due  to  the  Vietnam  conflict. 

Another  possible  reason  for  the  reduced  number  of  requests  for  retro¬ 
spective  searches  is  that  a  more  effective  SDI  program  is  operating.  In 
addition  to  the  Materials  Information  Bulletin  which  provides  a  Keyword  in 
Context  (KWIC)  title  list  of  new  AFML  reports,  an  SDI  program  has  been 
initiated  with  Chemical  Abstracts  Service  (CAS),  and  the  AFML  is  currently 
participating  in  the  NASA  SDI  program.  The  decrease  of  AFML  search  re¬ 
quests  is  an  area  that  shruld  be  investigated. 

A  listing  of  search  requests  by  subject  category  is  presented  in  con¬ 
junction  with  a  listing  of  documents  input  into  the  system  by  subject  category. 

This  parallel  listing  indicates  that  document  acquisition  is  reasonably  reflec¬ 
tive  of  the  desires  of  the  users  as  indicated  by  search  requests. 

Miracode  system.  The  AFML  maintains  a  Miracode  system  at  the 
Materials  Information  Branch.  This  system  operates  with  optically  coded 
16  mm  cartridge  microfilm.  Boolean  logic  operators  can  be  applied  in 
searching.  Several  auxiliary  systems  are  maintained  on  the  Miracode 
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Ths  University  provides  assistance  by  keypunching  item*  for 
eventual  coding  and  inclusion  in  the  appropriate  system.  The  Commerce 
Business  Dally  (CBD)  and  Independent  reaearch  reporta  are  two  systems 
the  laboratory  maintain.  Additionally,  the  computerized  AM  1C  document 
retrieval  ayatem  ia  duplicated  on  Miracode  *o  that  searches  arriving  at  the 
Materials  Information  Branch  requiring  immediate  reaponae  can  be  aearch- 
ed  on  aite.  The  search  capability,  however,  ic  net  aa  sophisticated  as  ia 
available  with  computer  searching. 

It  Is  expected  that  handbooks,  which  are  act  now  Included  aa  part  of 
the  AM  1C  ayatem,  will  be  aaaigned  apecial  access  numberf,  indexed,  and 
included  aa  a  apecial  Miracode  file.  Symposia  will  be  kept  at  the  AFML 
Library  and  retrieval  provided  by  a  KWIC  index  of  the  individual  papers  con¬ 
tained  in  the  symposia  proceedings. 

Thesaurus.  Several  changes  were  made  in  the  last  upaating  of  the 
thesaurus.  To  make  the  theaaurua  more  consistent  with  the  practice  of 
most  thesauri,  the  PO,  GT,  RT,  SA  designations  were  converted  to  Broader 
Term,  (BT),  Narrower  Term  (NT),  Related  Term  (RT),  and  also  see  (AS), 
respectively.  The  new  designations  have  the  additional  advantage  that  sort¬ 
ing  of  these  items  alphabetically  places  them  In  the  desired  order. 

A  very  useful  entry  Introduced  Into  the  thesaurus  is  the  "collection" 
term  which  serves  to  group  similar  items  under  a  common  heading.  These 
are  terms  which  are  not  hierarchically  related,  since  the  BT  and  NT  desig¬ 
nations  effectively  "collect"  the  hierarchically  related  items.  An  example 
of  such  a  collection  term  is  ADDITIVES.  This  term  shows  that  "additives" 
is  not  an  active  vocabulary  term  but  the  active  terms  which  could  be  consid¬ 
ered  additives  are  displayed  in  one  group.  The  entry  appears  in  the  thesaur¬ 
us  as  follow  i* : 


ADDITIVES  -  AS  INHIBITORS  INCLUDES  - 


CERAMIC  STABILIZERS 

0837550 

CHEMICAL  STABILIZERS 

0878000 

FUEL  ADDITIVES 

1554000 

LUBRICANT  ADDITIVES 

2169500 

PIGMENTS 

3171500 

PLASTICIZERS 

3193000 

POLYMER  FILLERS 

3248100 

PRESERVATIVES 

3314500 
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One  problem  which  has  tended  to  limit  vocabulary  and  thesaurus 
changes,  particularly  addition#,  ia  the  retronpective  updating  of  previous 
document*.  It  ia  essential  for  any  vocabulary  to  be  amenable  to  modifica¬ 
tion  to  accommodate  changea  in  technology  and  uaer  requirement#.  The 
AMIC  vocabulary  haa  already  undergone  considerable  modification  as  re 
ported  earlier.  7  However,  a#  the  vocabulary  haa  become  more  atable 
and  the  atore  of  document#  haa  increased,  there  haa  been  increaaing  reluc¬ 
tance  to  introduce  changes,  The  retrospective  updating  af  previous  docu¬ 
ment  Indexing  ia  at  beet  difficult  and  at  worst  impossible.  The  dilemma 
can  be  illustrated  by  the  following  question;  if  a  new  active  vocabulary  term 
ia  introduced  Into  the  system,  bow  can  all  previous  documents  which  should 
have  been  indexed  with  the  term  be  recalled  and  the  new  term  added  to  the 
indexing?  With  some  terms  there  i#  no  satisfactory  answer.  On  the  other 
hand,  the  decision  to  make  a  necessary  change  (i.  e. ,  as  shown  Ly  uaer  re¬ 
quirements  and/or  technological  evolution)  should  not  be  withheld  simply  be¬ 
cause  the  new  term  cannot  be  unilaterally  applied  to  all  previous  docum-nts. 

To  cope  with  the  problem,  it  was  decided  to  make  necewsary  changes 
(additions)  and  update  previous  documents  whenever  possible,  1 1  this  is  not 
possible,  the  date  at  which  the  term  becomes  effective  is  indicated  in  the 
thesaurus.  This  effective  date  informs  the  search  strategist  that  to  retrieve 
documents  previous  to  the  activation  of  a  term,  allowance  must  be  made  in 
the  search  strategy  for  this  contingency.  For  example,  MICROMECHANICS 
became  effective  1  July  1967.  To  retrieve  previous  documents  dealing  v/ith 
the  concept,  the  search  strategist  must  think  of  alternative  terminology 
which  might  have  been  used  to  express  it.  Terms  such  as  INTERACTIONS 
and  REINFORCEMENT  (matrix-reinforcement  interactions  to  express  micro- 
mechanics),  MICROSTRUCTURE,  PLASTIC  DEFORMATION,  and  RATE 
EFFECT  might  be  considered  in  the  search  strategy. 

American  Society  for  Metals  Search  Service.  The  University  of 
Dayton  had  been  participating  in  the  Research  Associate  Program  (RAP)  of 
the  American  Society  for  Metals  (ASM).  The  objective  of  the  program  was 
to  submit  searches  to  ASM  for  running  on  their  computerised  information 
retrieval  system  of  metallurgical  journal  literature.  The  preponderant 
interest  of  AFML  requestors  in  metallurgy  and  metallurgy  related  t.opico 
motivated  the  University^  participation  in  this  program  to  be  able  to  serve 
those  needs  more  effectively.  Despite  a  number  of  ASM  RAP  participants, 
the  service  proved  too  expensive,  and  satisfactory  arrangements  for  con¬ 
tinued  financial  support  were  not  forthcoming.  The  service  has  been  forced 
to  be  discontinued. 

Chemical  Abstracts  Service  SDI  Program.  To  provide  improved  in¬ 
formation  service  to  technical  and  professional  personnel  in  the  Air  Force 
Materials  Laboratory,  a  joint  effort  involving  the  Materials  Information 
Branch  (MAAM),  the  Polymer  Branch  (MANP),  and  the  Information  Systems 
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Section  of  the  University  of  Dayton  is  under  way  in  which  the  University  is 
participating  in  CAS  SDI  programs  for  the  benefit  of  selected  AFML  person¬ 
nel.  Three  types  of  document  abstracts  are  included:  SDI  Experiment, 
Polymer  Science  and  Technology  from  journal  *  (POST- J),  and  Polymer 
Science  and  Technology  from  patents  (POST-P),  A  description  of  these 
SDI  programs  is  given  in  Appendix  VI  in  conjunction  with  typiral.  user  profile*; 

Informal  comments  from  the  users  are  generally  fiivorable  regarding 
the  CAS  SDI  program.  The  Initial  success  achieved  and  the  acquisition  of 
more  experience  may  lead  to  an  expanded  endeavor. 

It  is  intended  by  CAS  that  the  SDI  programs  and  computer  tapes  be 
available  to  those  users  desiring  to  run  their  own  user  profile  searches. 

The  alternative  Is  for  CAS  to  run  the  programs  on  a  service  basis  with  u 
service  charge  dependent  on  computer  time  required.  The  programs  and 
tapes  should  be  capable  of  being  run  on  the  IBM  360  computer  Model  40 
acquired  with  the  advent  of  a  eooperati /e  venture  between  McCall  Corpora¬ 
tion  and  the  University.  The  SDI  program  will  be  run  at  the  University  to 
determine  the  cost  comparison  between  running  the  programs  and  the  ser¬ 
vice  charge  incurred  by  having  them  run  at  CAS.  If  it  is  shown  that  it  would 
be  more  economical  to  run  them  at  the  University,  this  procedure  would  be 
continued. 

Personnel  time  distribution.  The  time  spent  by  various  personnel 
ha*  been  broken  down  into  various  task  numbers.  The  task  number  defini¬ 
tions  are  given  in  Figure  4  and  the  personnel  time  distribution  for  various 
types  of  personnel  are  presented  in  Table  4. 
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FIGURE  4 


DEFINITION  OF  TASK  NUMBERS 

01  General  Includes? 

Supervision 
Meetings  &  trips 
Holidays  It  sick  leave 
Writing  of  reports 
Training  of  students 
Time  spent  with  visitors 
02  Input  Includes: 

Assignment  of  accession  numbers 
Document  accounting  records 
Preparation  of  index  and  abstract  cards 
Indexing 
Keypunching 

03  Output  Includes: 

Preparation  of  search  strategy 
Search 

Screening  of  searches 
Search  accounting  records 
Library  loan  functions 
Keypunching 

04  Updating  Includes: 

Review  of  vocabulary  and  thesaurus 
Changes  or  additions  to  previous  records 
Keypunching 

Acquisition  of  missing  documents 
05  (UD)  Research  Includes: 

Evaluation  studies 
Studies  of  new  techniques 
Investigation  of  new  systems 
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FIGURE  4  (Continued) 


05  (AFML)  Library  Include*: 

Preparation  of  Materials  Information 
Bulletin 

06  Special  Projects  Includes: 

Work  performed  in  support  of  the 
AFML  not  directly  related  to  AMIC 
retrieval  system 

07  Microfilming  includes: 

Time  spent  on  the  microfilming  of 
index/ abstract  records 
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TABLE  4 

DISTRIBUTION  OF  PERSONNEL  TIME  BY  TASK  NUMBER 


P rofesaionai  and  Clerical  at  UP: 

Task  Number  V»n  »nt  oi  'i  imr 


01 

23.  4 

02 

4b.  4 

03 

11.8 

04 

4.  3 

05 

1.8 

06 

10.5 

07 

0.  8 

Clerical  at  the  AF  Materials  Laboratory  Library 

Task  Number  Percent  of  Time 


01 

12.  3 

02 

49.0 

03 

7.9 

04 

3.  2 

05 

15.9 

06 

11.  6 

07 
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SECTION  V 


ESTABLISHMENT  OF  AN  INFORMATION  RETRIEVAL  SYSTEM  FOR 

INFORMATION  SCIENCE  RELATED  DOCUMENTS 

The  AFML  Library  ha»  received  a  number  of  Information  pcl“QCS 
documents.  These  document*  had  not  been  indexed  previously  h#.t «>.»■*  ail 
available  indexer*  were  required  for  indexing  material*  document*  far  the 
AMIC  document  retrieval  system.  It  was  agreed  bctwcou  the  AFML  and  the 
Univer*ity  that  the  inclusion  of  information  science  document*  in  the  AMIC 
document  retrieval  *y*tem  would  be  mutually  beneficial. 

In  the  *unmier  of  1967,  the  Department  of  Information  Science  wa» 
formally  inaugurated  at  the  University  of  Dayton.  The  Information  Sy*temt» 
Section  ha*  employed  a  graduate  student  from  the  Information  Science  De¬ 
partment  to  index  the  Information  *cience  related  document*.  The  training 
of  this  indexer  na#  not  been  nearly  a*  readily  accomplished  a*  the  training 
of  atudent*  for  indexing  material*  document*  (*ce  Section  D)  since  no  stand¬ 
ard  information  terminology  .tor  thesaurus  han  br*n  derived  for  information 
terminology.  Training  ha*  consisted  primarily  of  exposure  to  literature  on 
coordinate  indexing  and  attempt*  at  Indexing  with  a  review, 

Presently  'Tree  indexing",  i.e.  ,  *eiectiou  of  "index  term*"  without 
reference  to  a  predetermined  vocabulary  or  thesaurus,  if  being  performed 
■using  the  entire  document  a*  the  source  for  indexing.  Thi*  free  indexing  i» 
being  done  by  the  information  acience  tludent  and  being  reviewed  by  an  ex¬ 
perienced  indexer  (experienced,  however,  primarily  in  the  area  of  material* 
document*).  The  author*'  terminology  i*  retained  to  a  large  extent  unle** 
there  are  good  reason*  to  modify  or  »upplement  it.  A  group  of  «everal 
hundred  document*  will  be.  free  indexed  *ub*equent  to  which  the  "term*" 
will  be  keypunched,  sorted,  and  critically  reviewed.  This  review  will  be 
directed  toward  the  grouping  of  similar  concept*  with  relationships  of  terms 
(generic  or  hierarchical  relationships  a*  well  as  "related  term"  concepts) 
and  to  the  selection  of  actual  vocabulary  term*  which  most  appropriately 
define  and  express  the  concept  involved.  Thi*  procedure  wa»  the  one  follow¬ 
ed  at  the  beginning  of  the  University'*  involvement  with  the  AFML  in  the 
establishment  of  the  AMIC  document  retrieval  system^  and  the  technique 
was  also  applied  in  a  study  for  the  Air  Force  Flight  Dynamics  Laboratory.^ 

It  is  anticipated  that  the  terminology  of  information  science  may  present 
problems  since  information  science  is  a  newly  recognized  discipline  and  is 
generally  more  abstract  than  the  area  of  materials.  However,  the  experi¬ 
ence  acquired  from  previous  efforts  in  vocabulary  establishment  and  the¬ 
saurus  development  will  be  invaluable  in  deriving  a  useful  vocabulary  and 
thesaurus  from  the  free  Indexing  which  will  serve  as  the  basis  for  inclusion 
of  information  science  document*  in  the  AMIC  document  retrieval  system. 
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SECTION  VI 


SUMMARY 


Two  graduate  students  in  chemistry  and  a  high  school  physics  teacher 
were  employed  for  the  indexing  of  technical  documents.  Their  training  fol¬ 
lowed  closely  the  indexer  training  program  devised  the  previous  year  for  two 
chemistry  students.  Several  modifications  were  made  which  improved  train¬ 
ing  procedures.  It  was  found  that  de-emphasis  of  the  extracting  function,  i.  e. , 
writing  out  of  concepts  in  word,  phrase,  or  sentence  form,  was  helpful  in  re¬ 
ducing  indexer  training  time  and  in  permitting  consideration  of  the  document 
as  a  whole.  The  document  analysis  technique  was  found  to  be  very  important 
in  learning  the  indexing  function.  Modification  of  the  training  program  result¬ 
ed  in  a  reduction  of  the  formal  training  period  from  four  weeks  to  three  weeks, 
and  the  results  obtained  were  better  than  had  been  achieved  the  preceding  year, 
particularly  regarding  indexing  time. 

A  study  of  intra-  and  Interindexer  consistency  was  conducted.  Chi 
square  and  the  phi  coefficient  (see  Appendix  II)  were  the  statistical  techniques 
employed  to  determine  the  correlation  of  indexing  of  both  total  and  essential 
index  terms.  It  was  discovered  that  the  intra-  and  interindexer  correlation 
for  experienced  indexers  was  statistically  significant  at  p  >  0.999.  This 
high  degree  of  consistency  tended  to  verify  that  an  experienced  indexer's  in¬ 
dexing  can  serve  as  a  reference  standard  for  evaluating  the  work  of  indexer 
trainees.  The  agreement  of  essential  terms  was  markedly  better  than  was 
obtained  for  total  terms.  Using  one  indexer's  set  of  index  terms  as  a  refer¬ 
ence  point,  comparisons  were  made  of  the  performance  of  all  indexers  in¬ 
volved  in  the  study.  A  graphical  representation  (see  Figure  2)  shows  that 
with  experience  a  high  degree  of  consistency  among  indexers  can  be  expected 
under  the  conditions  provided,  i.  e. ,  materials  type  documents  as  an  index¬ 
ing  source  and  an  established  thesaurus. 

In  the  operations  of  the  document  retrieval  system,  it  was  noted  that 
the  number  of  technical  search  requests  declined  over  the  past  year.  A 
graphical  representation  of  the  search  requests  (see  Figure  3)  submitted  by 
both  the  AFML  and  ail  other  organizations  indicates  that  the  drop-off  is  at¬ 
tributable  primarily  to  reduction  of  use  of  the  system  by  AFML  personnel. 
Since  the  procedure  for  initiating  new  projects  results  in  search  requests 
for  the  document  retrieval  system,  the  usually  high  number  of  new  AFML 
projects  in  1965  and  1966,  as  compared  to  1967,  may  account  in  large  mea¬ 
sure  for  the  decline  in  AFML  search  requests.  More  effective  SDI  programs 
may  also  be  affecting  the  level  of  use  of  the  AMIC  document  retrieval  system. 
Minor  thesaurus  changes  were  effected.  The  ASM  retrieval  service  was 
forced  to  be  discontinued,  and  thus  is  no  longer  available  to  AFML  personnel. 
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An  SDI  program  with  Chemical  Abstracts  Service  was  initiated  for 
the  benefit  of  AFML  personnel.  SDI  Experiment,  POST-J,  and  POST-P 
are  the  CAS  SDI  services  subscribed  to.  Preliminary  results  are  encourag¬ 
ing,  but  profile  modifications  are  being  made  to  improve  the  service. 

The  Materials  Information  Branch  has  a  number  of  information 
science  documents  which  are  to  be  included  in  the  AMIC  document  retrieval 
system.  A  graduate  student  in  Information  Science  has  been  employed  for 
indexing  these  documents.  Free  indexing  is  presently  being  performed  from 
which  an  active  index  vocabulary  and  thesaurus  can  be  derived  which  will 
lead  to  inclusion  of  information  science  documents  into  the  system.  Assist¬ 
ance  in  establishing  an  information  science  system  may  be  forthcoming  from 
anticipated  contacts  with  other  organizations  involved  in  this  area. 
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SECTION  VII 


FUTURE  WORK 


A  new  concept  in  the  indexing  of  technical  documents  with  computer 
assistance  has  been  evolved  in  principle  and  will  serve  as  an  area  of  research 
for  the  coming  year.  The  ultimate  objective  of  the  research  is  the  develop¬ 
ment  of  a  computer-based  information  retrieval  system  which  can  be  accessed 
directly  by  the  user,  i.  e. ,  by  the  requestor  of  information  without  the  require¬ 
ment  ot  specialized  training  in  the  use  of  the  system. 

In  essence,  the  procedure  is  to  enter  the  entire  text  of  abstracts  (other 
portions  such  as  the  table  of  contents,  conclusions,  selected  specific  terms, 
etc.,  will  be  studied  later)  into  the  computer.  The  computer  will  order  the 
words  alphabetically  and  compare  each  word  with  two  word  lists.  One  word 
list  will  contain  nonsignificant  words,  e.  g.,  the,  and,  or,  studies,  etc. ;  the 
other  word  list  will  contain  technical  words  and  the  appropriate  disposition  of 
those  words.  This  technical  word  list  may  be  considered  to  be  a  computer 
contained  thesaurus  which  will  channel  all  technically  meaningful  words  to  the 
appropriate  active  vocabulary  terms  and  the  correct  generic  postings.  Words 
of  technical  nonsignificance  from  the  abstract  which  match  the  nonsignificant 
word  list  will  be  disregarded.  Words  which  do  not  match  either  list  will  be 
stored  and  printed  on  demand  for  decisions.  The  information  specialist  would 
then  have  the  assignment  of  deciding  if  the  r.onmatching  terms  should  be  enter¬ 
ed  into  the  nonsignificant  word  list  or  the  technical  term  list,  and,  if  it  is  to 
be  entered  into  the  latter,  how  should  it  be  entered,  what  generic  relationships 
exist,  what  related  terms  apply,  etc.  The  scheme  may  be  considered  as 
machine  aided  indexing  in  which  the  processing  capability  of  the  computer  is 
integrated  with  experienced  human  Intellect.  The  repetitive  operations  which 
are  required  in  human  indexing  would  be  assumed  by  the  computer,  while  the 
important  problems  of  term  relations,  semantics,  etc.,  would  be  concentrat¬ 
ed  on  by  the  information  specialist. 

On  the  output  side  of  the  operation,  a  request  stated  in  normal  text 
English  would  be  entered  into  the  computer  and  processed  in  the  same  way  as 
the  text  abstract,  i.  e. ,  with  nonsignificant  terms  disregarded  as  instructed 
by  the  nonsignificant  term  list  and  the  technical  terms  channeled  to  the  appro¬ 
priate  active  term  locations.  A  means  of  applying  Boolean  logic  to  the  terms 
will  be  devised  to  coordinate  terms  in  an  appropriate  sequence.  Interaction 
between  the  requestor  and  the  computer  would  take  place  so  searching  ;  nd 
retrieval  can  be  optimized. 

To  limit  the  scope  of  the  endeavor,  it  was  decided  to  concentrate  on 
the  area  of  composite  materials  and  to  consider  a  population  of  about  500  docu¬ 
ments,  large  enough  to  be  representative,  but  small  enough  to  be  amenable  to 


28 


treatment.  One  of  the  objectives  to  be  met  by  the  program  is  an  evaluation 
of  the  machine  aided  indexing  and  retrieval  system  by  comparing  it  to  the 
present  system  wholly  derived  from  human  indexing.  Another  objective  is  to 
indicate  the  feasibility  of  whole  text  processing  both  for  input  and  for  retrieval 
with  allowance  for  direct  requestor-system  interaction. 

To  carry  out  the  research  program  outlined  above,  various  hardware 
and  software  capabilities  must  be  established.  Considerable  software  must 
be  prepared  before  the  hardware  is  needed.  It  is  expected  that  a  remote 
Selectric  typewriter  terminal  to  the  IBM  360,  Model  40,  will  be  available  in 
the  offices  of  the  Information  Systems  Section  by  January  1968.  This  rather 
rudimentary  equipment  communicates  with  the  computer  character  by  char¬ 
acter,  but  this  equipment  should  prove  adequate  for  the  early  phases  of  the 
study.  By  the  spring  of  1968  it  is  hoped  to  have  a  cathode  ray  tube  (CRT) 
terminal  available.  This  unit  will  provide  much  greater  flexibility  in  display 
and  message  transmission  to  the  computer;  up  to  480  characters  can  be  ac¬ 
cumulated,  corrected,  and  transmitted  at  once  to  the  computer.  By  the  sum¬ 
mer  of  1968,  it  is  possible  that  a  remote  typewriter  unit  could  be  available 
at  the  Air  Force  Materials  Laboratory.  The  actual  installation  of  such  a  unit 
at  WPAFB  would  be  dependent  on  the  progress  made  at  UD  in  developing  the 
system  which  in  turn  will  determine  the  usability  of  the  system  by  users  at 
WPAFB. 
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APPENDIX  I 


BACKGROUND  INFORMATION  FROM  PREVIOUS  REPORTS 


The  original  system  established  by  the  Univeraity  w*a  *  coordinate 
Indexing  system  based  on  the  use  of  very  specific  terminology  and  the  utilisa- 
ti'jn  of  links  and  roles.  The  information  to  be  contained  in  the  system  was  a 
collection  of  approximately  10,000  acientific  and  technical  reporte  that  dealt 
with  all  aspects  of  materials  research  of  interest  to  the  Air  Force.  There 
was  no  predetermined  vocabulary  and  the  indexers  were  permitted  to  use 
whatever  terminology  that  was  needed  to  deaerib*  adequately  the  documents 
they  indexed.  The  first  five  hundred  documents  that  were  indexed  averaged 
over  eighty  link- role-term  combinations  per  report.  As  the  indexers  gained 
experience,  the  average  number  of  entries  was  gradually  reduced  to  slightly 
over  forty. 

An  attempt  to  build  a  thesaurus  of  the  system's  vocabulary  was  made 
during  the  second  year  of  operation.  At  that  time,  about  6,000  documents  had 
been  indexed  and  a  thesaurus  of  slightly  over  18,  000  terms  was  generated 
VVhen  the  system  was  updated  in  December  1963,  the  addition  of  4,  000  docu¬ 
ments  to  the  system  generated  30,  000  new  terms.  Although  the  system  had 
been  used  to  effectively  retrieve  information  in  answer  to  search  queries,  the 
rapid  growth  of  the  vocabulary  became  of  concern  since  it  was  desirable  at 
that  time  to  maintain  manual  searching  capabilities. 

The  results  of  an  evaluation  of  the  use  of  links  and  roles  in  the  system, 
and  the  rapid  growth  of  the  vocabulary,  formulated  the  basis  for  modifying  the 
system  during  the  fourth  year  of  operation.  Although  the  uie  and  value  of  role 
indicators  were  questioned  during  the  period  of  establishing  the  system,  a  de¬ 
cision  was  made  to  continue  their  use  until  the  system  was  operable  and  could 
be  formally  evaluated.  An  evaluation  was  made  after  the  system  became  oper¬ 
able  and  confirmed  the  opinion  that  the  use  of  roles  was  more  detrimental  than 
helpful  to  the  system.  The  use  of  roles  was  then  discontinued  and  that  elimi¬ 
nation,  in  addition  to  the  continual  growth  of  the  vocabulary,  necessitated 
modifications  in,  and  the  establishment  of,  vocabulary  controls. 

The  naming  of  materials,  primarily  metallic  alloy*  and  organic  com¬ 
pounds,  accounted  for  the  major  portion  of  the  rapid  and  apparently  unending 
growth  in  the  vocabulary.  Since  the  documents  that  were  indexed  into  the 
system  dealt  with  the  research  and  development  of  materials,  the  generation 
of  new  term#  was  anticipated  but  the  quantity  far  exceeded  the  expectations. 

The  approach  taken  to  establish  a  vocabulary  control  was  to  divide  the  vocabu¬ 
lary  into  classes  of  materials  for  analyses.  The  purpose  of  each  analysis  was 
to  determine  the  possibility  of  generalising  the  naming  of  materials  without 
losing  an  undue  amount  of  specificity.  Through  these  generalisations,  controls 
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were  established  for  the  naming  of  material!  and  the  system's  vocabulary 
was  reduced  to  approximately  10,000  term*.  Although  6,000  new  documents 
had  been  added  to  the  syatem  during  1965,  the  vocabulary  remained  fairiy 
constant.  The  use  of  the  coirt rolled  vocabulary  was  found  to  be  very  beneficial 
to  the  overall  operation  of  the  system. 

The  continued  growth  and  increased  searching  of  the  system  necessi¬ 
tated  additional  indexing  personnel.  Efforts  to  fill  these  positions  with  profes- 
sional  persons  were  unsuccessful.  It  was  then  decided  to  employ  students 
with  technical  backgrounds  to  index  technical  documents. 

Two  students  having  backgrounds  in  chemistry  and  one  physics  major 
were  employed  and  after  completing  a  training  program,  they  became  profici¬ 
ent  indexers.  The  training  program  was  specifically  designed  to  include  all 
aspects  of  information  retrieval,  although  the  main  emphasis  was  on  the  in¬ 
dexing  function.  By  being  exposed  to  the  overall  system,  the  students  acquir¬ 
ed  an  appreciation  of  the  importance  of  good  indexing.  The  students  were 
supervised  by  professionals  well  versed  in  indexing  and  the  other  aspects  of 
information  retrieval  and  having  similar  technical  backgrounds.  The  training 
program  was  flexible  to  allow  the  students  to  proceed  without  excessive  pres¬ 
sure.  The  students  were  expected  to  demonstrate  initiative  in  their  own  ad¬ 
vancement  as  indexers.  About  four  weeks  of  full  time  work  were  required  to 
complete  the  training.  The  training  program  consisted  of  the  following  steps: 

1.  Introduction  to  information  systems 

2.  Document  analysis 

3.  Extracting 

4.  Thesaurus  use 

5.  Exposure  to  screening  of  searches 

6.  Direct  indexing 

7.  Evaluation 

The  training  program  was  v/ell  accepted  by  the  students  as  evidenced  by  their 
written  comments.  They  felt  they  had  benefited  by  their  experience,  ft  was 
concluded  that  students  with  a  technical  background  are  capable  of  becoming 
proficient  indexers  by  applying  the  training  program  outlined  above. 

Several  thesaurus  changes  were  effected  in  the  thesaurus  updating 
(November  1966).  "SEE"  terms  were  changed  to  "USE"  terms.  A  number 
of  reference  terms  were  entered  'vhich  show  which  active  vocabulary  term(s) 
are  used  to  express  a  concept.  An  example  is:  HIGH  ENERGY  RATE  FORM¬ 
ING  USE  FORMING  153800  and  KATE  EFFECT  j408900.  Scope  note  type 
terms  were  added.  These  entries  explain  the  scope  encompassed  by  an  active 
term  or  define  a  term  if  a  definition  is  needed. 
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Date*  of  laaue  of  document*  were  included  a*  active  index  terms  ao 
that  aearching  by  date  ia  poaaible.  Subject  categoriea  were  eatabliahed  baaed 
on  Committee  on  Scientific  and  Technical  Information  (COSATI)  categoriea  to 
provide  data  on  the  aubject  diatri”  ution  of  new  acquiaitiona.  Thie  will  be  uae- 
ful  in  acquiaition  control  and  will  give  a  quantitative  meaaure  of  the  content* 
of  the  ayaiem.  Microfilm  printer- readera  and  microfiche  reader*  were  in- 
atalied  at  the  Univeraity  of  Dayton,  The  uae  of  thie  equipment  ha*  reaulted 
in  better  record*  and  greater  efficiency  in  operation. 
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APPENDIX  H 


CALCULATION  O*  PHI  COEFFICIENT  AND  CHI  SQUARE  FOR  INTRA-  AND 

INTERXNJEXER  CONSISTENCY 


Calculation  of  the  degree  of  association  of  judgment  of  essential  term* 

R  w«i  stcted  in  Reference  10  that  the  degree  of  correlation  between 
the  students'  indexing  and  the  supervisor's  indexing  »t«  better  with  regard 
to  eeeential  terms  »•  compared  to  the  correlation  of  total  terme.  It  wai  In¬ 
ferred  from  this  result  that  essential  terms  as  judged  by  an  experienced  in¬ 
dexer  represent  a  better  reference  standard  for  Indexing  evaluation  than  the 
total  index  terms  used.  Since  the  same  Indexer  indexed  the  same  documents 
one  year  later,  a  measure  of  the  reliability  of  the  indexer' a  judgment  could 
be  obtained.  The  criterion  for  establishing  the  reliability  of  the  experienced 
indexer’s  judgment  was  the  degree  of  association  or  correlation  between  the 
judgments  of  essential  terms  made  Independently  one  year  apart.  If  it  ran  be 
shown  that  there  is  a  statistically  significant  association,  then  one  has  evi¬ 
dence  for  the  verification  of  the  hypothesis  that  an  experienced  Indexer's  judg¬ 
ment  of  essential  terms  is  reliable.  The  particular  statistical  technique  chos¬ 
en  was  the  determination  of  the  <t  coefficient  anti  chi  square.  * 

One  factor  which  was  considered  was  the  possibility  that  the  judgment 
of  essential  terms  could  be  misleading  If  the  basis  of  the  study  were  chosen 
so  that  random  operants  might  cause  a  fortuitous  prea^sociation.  Such  an 
occurrence  would  detract  from  the  interpretation  of  the  results  unless  random 
p  re  association  were  accounted  for.  For  example,  suppose  a  document  v/ere 
indexed  with  five  terms,  four  of  which  were  Judged  to  be  easentirJL  by  an  ex¬ 
perienced  Indexer.  If  the  document  were  given  to  a  second  experienced  index¬ 
er,  and  the  second  experienced  indexer  selected  the  Sams  four  terms  as  essen¬ 
tial  terms,  the  d  coefficient  would  be  1.0,  indicating  perfect  correlation.  But 
by  chance  alone,  the  probability  of  the  first  term  being  an  essential  term  ia 
4/5.  The  probability  of  the  second  term  being  an  essential  term  Is  3/4.  The 
probability  of  the  first  two  terms  being  essential  is  4/5  *3/4=0.  60.  The  prob¬ 
ability  of  all  four  terms  being  randomly  selected  as  essential  terms  is  4/5* 
3/4*2/3*1/2=0.20.  Therefore,  there  is  some  chance  that  random  preassoci¬ 
ation  may  occur.  The  observed  data  indicated  that  the  average  document  in¬ 
dexing  consisted  of  about  20  terms  of  which  9  were  judged  essential.  The 
probability  that  all  9  essential  terms  would  be  selected  randomly  from  the 

list  is  calculated  as  follow#:  9/20*8/19 - --1/’2j9*  .0006,  This  value  Is 

so  close  to  0  that  for  ail  practical  purposes  it  may  be  Stated  that  random 
preassoci.at.ion  does  not  occur. 
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Observed  degree  of  association  of  essential  termi  -  Intraindexer  consistency 


To  test  the  reliability  of  judgment  of  essential  terms,  the  same  list  of 
terms  should  be  provided  to  Several  judges  (or,  in  this  case,  the  same  judge 
one  year  apart),  and  the  degree  of  association  determined.  It  was  observed 
that  about  45%  of  the  terms  were  judged  to  be  essential  by  FL£  both  in  1966 
and  1967.  If  the  judgments  were  perfect,  for  a  typical  document  of  20  term#. 
9  terms  would  have  been  judged  essential  in  1966  and  the  Same  9  terms  would 
have  been  judged  essential  in  I9b7.  a  wit  shown  above  that  there  is  no  ex¬ 
pected  significant  association  of  terms  if  9  terms  are  randomly  assigned  out 
of  a  possible  20  termi  for  two  different  judgments.  In  actual  fact,  the  lists 
of  terms  to  be  judged  were  not  identical.  The  number  of  terms  judged  essen¬ 
tial  wa#  about  7%  higher  in  1967  than  in  1966.  The  total  number  of  terms  us¬ 
ed  and  hence,  forming  the  basis  for  judgment,  was  about  20%  higher  in  1967 
than  in  1966.  If  one  can  assume,  however,  that  the  total  number  of  terms 
used  in  indexing  does  not  appreciably  affect  the  judgment  of  whether  or  not  a 
term  is  essential,  an  assumption  already  inherent  in  the  concept  of  essential 
terms,  one  can  determine  the  average  number  of  total  terms  used  for  index¬ 
ing  the  series  of  35  documents  in  the  two  years.  This  figure  can  be  used  for 
calculating  the  &  coefficient  and  for  obtaining  the  degree  of  association  of 
the  essential  terms  over  the  two  years.  It  Is  necessary  that  the  same  number 
of  total  terms  be  used  in  the  calculations.  It  should  be  noted  that  the  actual 
essential  terms  obtained  were  used.  The  total  terms  for  1966  and  1967  were 
adjusted  to  the  average  for  calculation  purposes.  Since  the  number  of  essen¬ 
tial  terms  of  the  two  years  is  not  equal,  the  maximum  possible  &  coefficient 
is  less  than  1.0.  The  Of  coefficient  obtained  for  the  association  of  the  essen¬ 
tial  terms  is  a  good  indication  of  the  reliability  of  the  judgment  of  essential 
terms. 


1966 


Observed  data 


1967 


Non,  Ess. 

Ess. 

Ess, 

(A) 

70 

(B) 

191 

Non  Ess. 

(C) 

247 

(D) 

90 

317  281 


261 

337 

598 


d  =  BC  -  AD 

V( A+B)  (C+D)  (A+C)  (B+D) 


47177  -  6300 

V  (261)  (337)  (317)  (281) 
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d  -  0.461 

X2  =  NO12 

*  598  (.461)2 

-  127 

ihii  value  of  X  is  significant  at  p  >  ,99°.  Therefore,  a  high  degree  of  as¬ 
sociation  is  indicated. 

Maximum  possible  degree  of  association  of  essential  terms 

The  maximum  possible  degree  of  correlation  can  be  determined  by- 
considering  the  hypothetical  case  of  all  terms  considered  essential  io  1966 
also  being  considered  essential  in  1967.  Referring  to  the  table  derived  from 
the  observed  data,  one  can  see  that  Cell  B  must  now  be  261  since  all  essential 
terms  from  1966  are  considered  essential  also  in  1967.  Cell  A  must  be  0 
since  there  were  no  terms  considered  essential  in  1966  that  were  not  also 
considered  essential  in  1967.  There  were  actually  261  term*  considered  es¬ 
sential  in  1967.  Of  these,  261  match  with  1966  essential  terms  leaving  20 
which  were  considered  essential  in  1967  but  not  in  1966.  Therefore,  Cell  D 
must  be  20.  Accounting  for  the  remaining  terms,  there  were  598  total  terms 
in  1966;  261  were  essential  leaving  337  nonessential.  However,  of  these  337 
nonessential  terms,  20  were  considered  essential  in  1967  and  are  located  in 
Cell  D  leaving  317  terms.  In  1967  there  were  598  terms  of  which  281  were 
essential.  The  essential  terms  are  accounted  for  in  Cells  B  and  D.  There 
were  no  noneseential  terms  from  1967  considered  essential  in  1966  as  noted 
earlier;  this  condition  is  indicated  in  Ceil  A  as  0.  The  remaining  terms  are 
nonessential  terms  and  are  calculated  as  598  -  281  =  317.  This  figure  matches 
the  figure  of  nonessential  terms  from  1966  and  is  the  value  for  Cell  C. 


a 

max 


1967 


Non  Ess. 

Ess. 

1966 

Ess. 

(A) 

0 

(B) 

261 

261 

Non  Ess. 

(C) 

317 

<D) 

20 

337 

317 

281 

598 

=  BC  - 

AD 

= 

96831  -  0 

V(A+B)  (C+D)  (A+C)  (B+D)  V  (261)  (337)  (317)  (281) 


&  =  0.934 

max 
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Obaerved  degree  of  aaaociation  of  caaential  terma  -  Interindexer  conalatency 


If  the  concept  of  essential  term*  is  of  value,  there  must  he  some  de¬ 
gree  of  r-oneiatency  between  different  Individual*  a*  well  a*  with  the  lime 
individual  at  different  tlmaa.  Applying  the  fame  assumptions  a*  for  deter- 


the  Inter iud 0  woculcieiit  wai 


ed  for  two  experienced  indexers, 

ubaerved  data 


LQ I  (1967) 


(—  - 

Non  Ess. 

Eaa. 

FLS 

La  a. 

(A) 

78 

(B) 

203 

281 

(1967) 

Non  £ii. 

(C) 

249 

(D) 

160 

409 

327 

363 

690 

=  BC 

AD 

S 

60547  -  12480 

VCA  +B)  (C+D)  (A  +C)  (B+D)  V (£81)  (409)  (327)  (363 ) 

d  =  .327 

X2  =  N(2Z 

=  690  (,  327)2 
=  73,8 


Thia  value  of  la  significant  at  p  >  .999. 

Maximum  poaaible  degree  of  aaaociation  of  eaaential  terms 

The  maximum  poaaible  degree  of  aaaociation  was  determined  for  in¬ 
terindexer  conaiatency  in  the  aame  way  as  for  intraindexer  consiatency  which 
Was  deacribed  previoualy. 


FLS  1967 


LCH  1967 


Non  Eaa, 

Eaa, 

Eaa. 

(A) 

0 

(B) 

281 

Non  Ess. 

(C) 

327 

_ 

(D) 

82 

327  363 


281 

409 

690 
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BD 


AC 


94887  -  0 


4 

V(A+B)  (C+DJ  (A+C)  (B+D)  V(Z8l)  (409)  (327)  {36T) 

(J  *  0.814 
max 

Cornparing  <t  coefficient*: 


a 

lntralndexer 

*  0.461 

Gt 

max 

=  0.934 

a 

interindexer 

=  0.327 

a 

max 

-  0.814 

Therefore,  the  degree  of  ••■delation  between  essential  term*  1*  significant 
regarding  both  lntralndexer  and  Interindexer  reliability,  although  intra- 
indexer  reliability  i*  better.  The  value*  are  shown  graphically  in  Figure  1, 
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APPENDIX  III 


TYPICAL  INDEX  CARDS  FOR  ALL  INDEXERS  INDEXING  THE  SAME 

DOCUMENT  (Ref,  5)  (29,748) 


TITLE;  Uniform  Vacuum  Ultraviolet  Reflecting  Looting*  on  Large  Surface* 

ABSTRACT:  A  method  for  producing  uniform  vacuum-deposited  filma  on 
large  aubatratei  from  a  single,  until  area  evaporation  aource  ia  described. 
Th«  prf>vctiu rp  is  Wing  applied  to  the  deposition  of  a  uniiorro  coating  of  mag¬ 
nesium  fluoride  on  a  39-inch  diameter  vacuum -deposited  aluminum  mirror 
for  use  In  the  far  ultraviolet.  The  magnesium  fluoride  reetrlcta  the  oxidation 
of  the  aluminum  and  a  uniformly  high  reflectance  surface  ia  obtained,  Reaulta 
of  teata  on  a  large  number  of  two-inch  aquare  aamplea,  arranged  to  duplicate 
the  sixe  and  paraboloidal  ahape  of  the  mirror,  indicate  that  a  reflectance  of 
82  +  2  percent  at  Lyman  alpha  can  be  expected. 
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FLS  (19b6) 


F LS  (19b7) 


LCH  (1967) 


♦**R  r  flrctlve  coatings 
Vacuum 

**♦  Magneiium  fiuoride 
♦♦♦Aluminum 
Coating 
’•'••Mirror  s 

♦•♦Ultra violet  radiation 
♦♦♦Reflectance 
Measurement 
♦♦♦Vapor  deposition 
Surfaces 
Films 


CODE: 

*  Term  judged 
*♦  Term  judged 
**♦  Term  judged 


*♦♦  Reflective  coatings 
♦♦♦Reflectance 
♦♦•Ultraviolet  radiation 
♦♦♦Vapor  deposition 
Film  ■ 

♦♦♦Magnesium  flworlrln 
+  **A1  umlnum 
♦♦♦M  ir  r  or  a 

Aluminum  coatings 
Wavcle  ngth 
Protective  coatingB 
Coating 

Manufacturing  technology 
Surfaces 


♦♦♦Vapor  deposition 

+  *  wp  *»t  I  urMi/j. 

♦♦♦Magnesium  fluoride 
♦♦♦Aluminum 
Vapor  phase 
♦Vacuum  chambers 
♦Films 
Size  effect 
♦Apparatus 
Evaporation 
♦Aluminum  coatings 
••♦Reflectance 
Mea  turement 
♦♦♦Mirrors 
Flow 

♦  ♦♦Ultraviolet  radiation 
Evaporator  s 
Design 

Theoretical  analysis 
Coating 


essential  In  one  set  of  index  terms 
essential  In  any  two  sets  of  index  terms 
essential  in  any  three  sets  of  Index  terms 
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DCS  (1906) 


PA  T  (196?) 

kj  4/1.. _ IJ_ 

uaui 

Vapor  deposition 

Aluminum 

Mirrors 

Antioxidants 

Reflective  coatings 

Reflectance 

Optical  properties 

Ultraviolet  radiation 

Vac  uum 

Wavelength 

Design 

Coating 

Measurement 

Fabrication 

Theoretical  analysis 

Central  systems 

Testing  methods 

Apparatus 

Optical  instruments 

Surfaces 


Magnr  sium  fluoride 
Protective  coating* 

R  eflectance 
Ultraviolet  rsdistlnn 
Aluminum 
Vapor  deposition 
Aluminum  coatings 
Space  vehicles 
Reflective  coatings 


TRD  <19f>6j 

Vapor  deposition 
Films 

Magnesium  fluoride 
Mirrors 

Aluminum  coatings 
Ultraviolet  radiation 
Space  vehicles 
R  eflectance 
Reflective  coatings 


EJF  (196?) 

Reflective  coatings 
R  eflectance 
Varuum 

Ultraviolet  radiation 
Aluminum  coatings 
V apor  deposition 
Films 

Testing  methods 
Magnesium  fluoride 
Mirrors 

Theoretical  analysis 
Fabrication 
Protective  coatings 
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TABLE  •>.  FREQUENCY  DISTRIBUTION  OF  INDEX  TERMS 

No.  of  Indexers 
Using  the  Term 


Magnesium  fluoride  7 

Reflectance  7 

Reflective  coatings  7 

Ultraviolet  radiation  7 

Vapor  deposition  7 

Mirrors  6 

Aluminum  5 

Aluminum  coatings  5 

Coating  5 

Films  5 

Protective  coatings  4 

Measurement  3 

Surfaces  3 

Theoretical  analysis  3 

Vacuum  3 

Apparatus  2 

Design  2 

Fabrication  2 

Space  vehicles  2 
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TAJ  L£  5.  (Continued) 


TERM 


No.  of  Indexer* 
Using  the  Term 


Te  tig  method*  2 

Wavelength  2 

Antioxidants  1 

Control  systems  1 

Evaporation  1 

Evaporators  1 

Flow  1 

Manufacturing  Technology  1 

Optical  Inst  ruments  1 

Optical  properties  1 

Size  effect  1 

Vacuum  chambers  1 

Vapor  phase  1 
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APPENDIX  IV 


SEARCH  REQUESTS  PROCESSED 
December  I S* 66  to  December  1967 

drirch 


No, 

Requestor 

Organization 

Search  Title 

M.D.  Ward 

Hercules  Powder 

Co.  Utah 

Failure  Criteria  for  Alloy# 

88.7 

ST  INFO 

WPAFB 

Hydr,  Lnbr.  System  Contaminating 

88V 

Mr,  Bufgina# 

Lawrence  Radiation 
Lab. 

Strength  Degradation  of  Fii,.  wd. 
Comp.  Long  Time  load 

890 

J.  Sullivan 

WPAFB 

Hydrol.  Stab,  of  Slloxane  or 

Silicate  Hydraul  FI. 

891 

F.  Meyer 

WPAFB 

Corroalon  of  Ti  in  Organic  Acids 

8;. 

ST  INFO 

WPAFB 

Chromium  Alloy  for  Turbine  Blade* 

89.' 

lit.  Carey 

WPAFB 

Radiation  from  Projectile  Impact 

894 

F.  Meyer 

WPAFB 

Protective  Coatings  for  Superalloy 

895 

F.  Meyer 

WPAFB 

Corrosion  of  Metal  Composites 

896 

E.  J.  Erwood 

Xanthippe  Chemicals 

Electrically  Conduct.  Greases 

897 

D,  Wetzler 

AFC  Industries 

Titanium- Al-V  Effect  on  Iron  on 
Properties 

898 

D.  Wetzler 

AFC  Industries 

Springs  of  Titanium- Al-Cr-Fe- Mo 

899 

H.  Smith 

Boeing  Airplane 

Mechanical  Properties  of  Germanium 

90' 

J.  Charleaworth 

WPAFB 

Measurement  Testing  of  Thermal 
Properties 

901 

F.  Scheffler 

UD 

Organic  Semiconductors 

902 

K.  S.  Mazdigasni 

WPAFB 

Decomposition  of  Chelates 

903 

R.  Hanachka 

Hamilton  Standard 

Fatigue,  Creep,  Tensile  Prop,  on 
Aluminum  6061 

904 

G.  A.  Gegel 

WPAFB 

Atomization  of  Liquids,  Metals 

905 

R.  Wood 

Aerojet  General 

Oxidation  Protective  Coatings 

906 

Dr.  A.  Prince 

WPAFB 

Mean,  of  Electrical  Conductivity 
of  Solutions 

907 

R.  G.  Prince 

Straza  Industries 

Optical  Properti  j  of  Invar. 

908 

M.  Manoff 

WPAFB 
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Olin  Reeearch  Center 

Polymer*  Since  1967 

1137 

F.  Oatermann 

WPAFB 

Fatigue  S<  Heat  Treatment  of  Al-7075- 
T73 

1138 

J.  Stock 

Fairchild  Hiller 

Mech.  Prop,,  Toughne**  of  Ti-Al- 
Sn-V 

1139 

C,  A,  Lombard 

WPAFB 

Oxidation,  Ductility,  Str  of  Tl  Alloy* 
at  HI  Temp*. 
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.'r* ^  r rn 


No. 

R  *  nii  m  m  t  ty  r 

O  rganl  rsti  mi 

Search  Title- 

1  I -to 

P .  Thomas 

irn 

l,.f - A  C - a  _  - - -  _  /  /—  _  ,i _ _  ,i 

■  i'  *-  A  -■  7  Vi  V-&1  wyAyHL 

Acid  -Salt? 

1141 

F ,  F  rui  n 

UP 

Boron  Hydride*  Anion  Molecular 
Structure 

1  142 

U.  Buss 

WPAFE 

Fluidic  Controls  for  Gas  Turbines 

114^ 

A.  Heidernan 

Lockheed  Calif.  Co. 

Fatigue  of  Steel*  and  T1  Alloy* 

1  1  44 

A.  Heidtrmitn 

Lockheed  Calif.  Co. 

Fitigu**  Data  iwi  Tf  Alloys 

1 1 4  S 

C.  Bulenca 

UD 

Graphite  Fiber  Surface  Treatment 

\  l  46 

C.  Eolenca 

UD 

Surface  Treatment  of  Fiber*  for 
Composites 

1147 

T.  Reinhart 

WPAFB 

Coating*  or  Band*  for  Projectile* 

1148 

S.  Zakanycr 

WPAFB 

Mai*  Dtffu»lvity  of  Helium  and  Air 

1  149 

J.  Charleeworth 

WPAi'B 

Melting  Point*  of  Non  Metalo 

1150 

J.  Charieswurtli 

WPAi'B 

Melting  Point  of  Metal*  it  Alloy* 

1151 

J.T.  O’Brien 

WPAFB 

Flammability  of  Nonmetalllc  Fiber* 

1  152 

R.  Dominic 

UD 

Fatigue  of  AI-7075-T6 

1153 

S.  L.  Heulein 

US  Naval  Ord.  Lab. 

Fracture  and  Strese  of  Gla*» 

1154 

S,  L.  Hanlein 

US  Naval  Ord.  Lab. 

Handbook*  Si  Symposia  for  Acoustical 
Prop. 

1  155 

USAF  HQ*. 

USAF  HQ 

Sierracoat  RS  Antireflectlve  Coating* 

1  156 

M.  Guenther 

WPAFB 

Hydrostatic  Forming 

1157 

W.  Jater 

WPAi'B 

Sublimation  of  Brasses 

1  158 

T.  Dolce 

WPAFB 

Ph enolic- Refr asil  Decomposition 

1159 

F.  E.  Fro*t 

Lawrence  Radiation 
Laboratory 

Mechanical  Prop,  and  Behavior  of 

F  oam* 

1160 

P,  Monaghan 

WPAFB 

Catalyst*  for  Synthesis  of  CO-> 

1161 

P.  Monaghan 

WPAI’B 

Cathode  Sputtering 

1  162 

L,  Par*on* 

WPAFB 

Fatigue  of  Xncouel  718 

1163 

W,  Scardino 

WPAFB 

Bonding  or  Repair  of  Metal  Plastic 
Struc, 

1  164 

I\  SchefHer 

WPAFB 

Composites  and  Laminate* 

1 165 

E.  Pieracy 

Howmet  Corp, 

Diffusion  Coating*  for  Superalloy* 

1166 

G.  E.  Metxger 

WPAI’B 

Joining  uf  Titanium 

1 167 

A,  Olevitch 

WPAI'B 

Outga»uing  in  Vacuum  or  Oxygen 

1 168 

A.  Olevitch 

WPAFB 

Fire  Retarding  Coating  or  Plastic* 

1 169 

j.  Rhodehamel 

WPAFB 

Low  Temp,  Curing  Polymer  Matrix 

1170 

E.  Mat*on 

Goers  Optical  Co, 

Splicing  of  Mylar  and  E*tar 

1172 

R.  Spain 

WPAFB 

Mechanical  Properties  of  Porou.* 
Polymeric  Fibers 

Ml  59 

Lt.  Peyton 

W  PAFB 

Compatibility  of  H2O2  with  Zinc 

Pow,  Paints 
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REQUESTOR  INDEX 


Abshire,  E.  Albany  Metallurgy  Research  Center/Oregon 

1065,  1126 

AFAPL/Stinfo  APP,  APF,  APFG,  APFL,  APFT,  API,  APIE,  APIP, 

A  PIT,  APR,  APRC,  APRP,  APT,  APTC,  APTP, 
APM,  APMD 

888,  892 


Anderaon,  C.  S. 

1080 

MATC 

AFML 

Aponyi,  T.  J. 

1067,  1087 

MANC 

AFML 

Askins,  R. 

1112,  1116 

UDRI 

Bailer,  Mr. 

1046 

MATE 

AFML 

Bair,  F.  H. 

1098 

MAAA 

AFML 

Baas,  Lt. 

1142 

MAAM 

AFML 

Berner,  W. 

912 

UDRI 

Bidula,  R. 

967 

MAMS 

AFML 

Bolenca,  C. 

1145,  1146 

UDRI 

Booth,  Mr. 

944 

Turco  Products 

Bowman,  R.  E. 

955 

MAMP 

AFML 

Bradstreet,  S. 

1128 

Dayton, 

Ohio 

53 


Cro«by,  J. 
931 


MAMP 


AFML 


Cutrell,  F, 
1122 


Martin/ Denver 


■J* 

} 

I 

Dahl,  A,  F.  Bruue'wick  Corp*.  * 

920 


Dalton,  A.  S,  MAAS  AFML 

1035 


Davie,  S.  M  A  AM  AFML 

988 


DePierre,  V.  MAMN  AFML 

1123 


Dimiduk,  R.  W.  MAYT  AFML 

1018,  1058 

Dole®,  T.  Aberdeen  Proving  Ground® 

1158 

Dominic,  R,  UDRI 

1152 

Donlan,  V.  L.  MAYH  AFML 

1021,  1061,  1063 

Doyle,  H.  Boeing/Vertol  Divieion 

930,  932 

Eichelman,  G.  E.  MATC  AFML 

1033 


Em  rich,  B.  MAAM  AFML 

954,  1064 

^"vood,  F.  J.  Xanthippe  Chen-' 

896 


Faller,  Dr.  J.  G.  Boeing/Vertol  Divid  on 

934 


Fearn,  R.  A,  C.  Babcock  and  Wilcox,  RfcD  Divieion 

1001 


Felker,  T.  S.  MATB  AFML 

102? 


55 


Ferguson,  J 
985 


APFG-36  AFAPL 


Fouse,  G.  T.  AVP  AFAL 

1050 

Frederick,  W.  G.  D.  MAYT  AFML 

1019,  1023 

Frost,  F.  E,  Lawrence  Radiation  Lab. 

1159 

Fruin,  E.  UDRI 

1141 

Gegel,  G.  A.  MATB  AFML 

904,  1032,  1040 

Gegel,  Dr.  H.  L.  MAMS  AFML 

1056 

Graves,  B.  UDRI 

917 

Greene,  M.  I.  American  Cyanamid  Co. 


Greeson,  R.  Beech  a/c  Co. 

924,  925 

Guenther,  M.  A.  MATF  AFML 

1156 

Hanlein,  S.  L.  US  NOL 

1153,  1154 

Hanschka,  R.  Hamilton  Standard 

903,  918,  949,  950 

Harmsworth,  C.  L.  MAAM  AFML 

983,  1106 

Heaver,  R.  A.  Racine  and  Vickers  Armstrong,  Inc. 

1117 

Heideman,  A.  Lockheed  California  Co. 

1124,  1125,  1143,  1144 


Hemenger,  P.  M. 

1017 

MAYE 

AFML 

Henderson,  R. 

1129 

MAAS 

AFML 

Herman,  R. 

1110 

Airesearch  Mfg.  Co. 

Hi  ch elm  an,  G.  H. 

979 

MATE 

AFML 

Holehouse,  I. 

1015 

MAAM 

AFML 

Hopkins,  A.  K, 

1062 

MAYR 

AFML 

Horne,  E. 

1130 

MAAM 

AFML 

Horsfield,  Capt. 

1025,  1037 

MATE 

AFML 

Huie,  R. 

986 

APFT-30 

AFAPL 

Immel,  L.  C. 

938 

Arthur  D. 

Little,  Inc. 

Jajudi,  Mrs. 

990 

Olin  Research  Center 

James,  Capt.  D.  R. 

939,  940,  941, 

MAAE 

942,  943, 

AFML 

960,  961, 

Jater,  W. 

1157 

Johnson,  W.  P. 

1013 

MATC 

AFML 

Jukkola,  Dr. 

MAAA 

AFML 

936 

Karnowaky,  M.  M,  Sandia  Corp. 

933 


1115 


57 


nzsteis.  j . 

769,  970 

MAMs 

AFML 

Keenerd,  h,  l,, 

1079 

MATS 

AFML 

Kopell,  I,. 

1069 

MATC 

AFML 

Lichrninn,  Lt,  H. 

928 

MAAM 

AFML 

Larsen,  R. 

935 

EGIG  Inc. 

Lawson,  W. 

1024 

Pornpano  Beach 

Liaker,  H. 

1012 

Avco  Corp. 

Utvak,  S. 

1068,  1077 

MATC 

AFML 

Lombard,  C.  A. 

921,  1139 

MAMP 

AFML 

Love,  K.  L. 

1044.  1048 

MATB 

AFML 

Mahieu,  W. 

1135 

UDJU 

Manoff,  M. 

908 

MANC 

AFML 

Matson,  E. 

1170 

Goers  Optical  Co. 

Maadlgasnl,  K.  S. 

902 

MAJMC 

AFML 

McClintsck  F 

948 

LET 

McConnell,  B.  D. 

MANL 

AFML 

1097 


53 


McIntyre,  W.  R, 

965,  1059 

Tinker 

A  KB 

McKelvay.  E.  W. 

982 

MAAS 

AFML 

Meek*.  A.  C. 

1  u>9 

MALT 

AFML 

Membry,  T, 

1002 

Singer 

Co. 

Meraal,  S. 

927 

MAMC 

AFML 

Met.'.ger,  G.  E. 

1134,  1166 

MAMP 

AFML 

Meyer,  F, 

891,  894,  895, 

MAAS 

937,  946, 

AFML 
959,  978 

Miller,  E.  MATE 

1011,  1036,  1038 

AFML 

Mittendorp,  H, 

992,  993,  994 

MAMP 

AFML 

Monaghan,  P. 

1160,  1161 

AVTL 

AFAVL 

Morrieeey,  E, 

1051,  1093 

MAAE 

AFML 

Muntt,  J,  H. 

1016 

MAYA 

AFML 

O'Hara,  Lt.  W.  T.  MATB 

1042,  1043,  1074 

AFML 

Olevitch,  A. 

1167,  1168 

MAAE 

AFML 

OitirmaoD,  F. 

MAMD 

AFML 

1137 

Packard  Hewlett  Packard  Hewlett 

1095 


59 


:  i  h.i 


!■*  J  .  ATI.. 

1  100,  1107, 

M  *  r 

1119,  1120 

At-  M  1. 

Payer,  I. 

?6e 

MAMS 

AFML 

Peyton,  Li.  K.  S. 

Mi  59 

MAAS 

AFM  I, 

Phillipi,  C.  M. 

1022 

MAYA 

A  PM!. 

Phillip*,  L. 

909 

AFFDL 

Pleracy,  E. 

1165 

Howmct 

Corp. 

Pitt*,  Li.  Col,  J.  W. 
966 

MAAS 

AFML 

Poliqnin,  P. 

1031,  1049 

MATE 

AFML 

Polley,  L.  C. 

1039,  1041 

MATS 

AFML 

Prince.  Dr.  A. 

906 

Purcell,  Capt.  G. 

1054 

MAAE 

AFML 

Racca,  R. 

913 

Nichola  Engineering 

Ray,  J.  D. 

mss,  looo,  .• 

MANC 

>90 

AFML 

Reiman,  W. 

974,  975,  976 

MAMD 

AFML 

Reinhart,  T. 

972,  973,  991, 

MAAE 

1147 

AFML 

60 


MAMS 


AFML 


Rennat,  H.  O. 

947 

Rhodehamsl,  J.  MAC  AFML 

1169 

Roelefs,  H.  Interstate  Steel  Corp. 

926 

Rolinski,  E.  J.  MAYT  AFML 

1060 

Rooney,  W.  L.  MAAE  AFML 

1052,  1094 

Rosenburg,  Dr.  H.  MAYH  AFML 

962 

Rose,  J.  H.  MANF  AFML 

1004,  1005 

Ruben,  B.  MAAA  AFML 

1102 

Russo,  Capt.  V.  MAAS  AFML 

1091 

Ryan,  M.  T.  MAYA  AFML 

1020 

Scardlno,  W.  M.  MAAE  AFML 

915,  1163 

Scheffler,  F.  L.  UDRI 

901,  914,  916,  922,  923,  1092,  1164 

Schmidt,  D.  L.  MANC  AFML 

1081,  1082,  1083,  1084,  1085,  1086 

Schulz,  Capt.  W.  J.  MAAE  AFML 

984 

Schwartz,  H.  S.  MAN  AFML 

919,  971 

Schwartz,  J.  H. 

1136 


Olln  Research  Center 


.*?(  ,’iw  n  rt  a,  I 
100  3 

Schwartz,  L.  H. 

929 

Shoemaker,  Mr, 

not 

ijUtyw,  J,  K. 

10U 

Siiha,  Lt.  C.  W. 

98? 

Slenk,  C. 

963 

Smith,  E.  S. 

1072 

Smith,  H. 

899 

Smith,  R.  G. 

907 

Smoot,  P, 

998,  999,  1000 

Spat  a,  11. 

1172 

Stanley,  Mr. 

7 1 1 

Starr,  E. 

1099 

Stelraon,  D. 

952,  997 

Jtock,  J. 

1138 

Strnat,  Dr. 

957 


k  aeo  Research 

M*nL«ba , 

Coata  and  Clark  Inc. 

MAKE  AJFML 

APET-29  AFAPL 

Carborundum  Co. 

MATF  AJFML 

Boeing  Airplane  Co. 

Straxa  Industries 

Army  Material  Research 
MANC  AFML 

Martin/Orlando 

Warren- Law  aon 

MAAE  AFML 

Fairchild  Hiller 

MATE  AFML 


62 


Sullivan,  J, 

S90 

MAAF 

A  Ku 

Ta~i«,  C, 

1075.  1076 

MATC 

AFML 

Thpniai  P  _ 

»fr\v.v 

vUru 

1 1 40 

Trlckeit,  G.  W. 

1 07  J 

MATR 

AFM  L 

TrinVle,  Mr. 

1045,  1070 

matl 

AFML 

Udlock,  R.  H, 

1104 

General  Dynamic* 

USAF  Headquarter* 
1155 

USAF  Headquarter* 

Van  VUet,  R,  M. 

1101 

MAYT 

AFML 

Ward,  M.  D. 

887 

Hercule*  Powder  Co, 

Wateon,  R, 

1105,  1111 

Iowa  State 

Unlveraity 

Webb,  J. 

958 

AFAFL 

Weidner,  J. 

1127 

UDPI 

Wetaler,  D. 

897,  898 

A,  F.  C.  Induatrle* 

Waxier,  M. 

995,  996 

General  Electric  MSD 

Wheeler,  W.  H. 

1103 

maae 

AFML 

Whitney,  F.  L. 

1053 

MATE 

ATML 

6  i 


Wilkinson,  K,  F. 

t  M  u 

'  •  ■  u 

Williams,  C. 

1131 

Wood,  R. 

90b 

Wurst,  J, 

980,  981,  989 

V*rosi,  A, 

910 

Young,  G. 

945 


Clary-  Corp, 
Bridgeport  Brats  Co. 
Aerojet 

UDKI 

5ESECI  AFSC 
MAAM  AFML 


Zakanycz,  S. 

951.  1132, 


ASBED-40  AFSC 
1133,  1148 
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INDEX  OF  REQUESTING  ORGANIZATIONS 
Report  for  Period  I  December  1966  to  1  December  1967 


Organization 

Aberdeen  Proving  Grounds 

Aerojet  General 

AFAL 

AFAPL 

AFC  Industrie* 

AFFDL 

AFML 

AFSC 

Airesearch  Mfg.  Co. 

Albany  Metallurgy  Reaearch  Center 
American  Cyanamid  Co. 

Army  Material  Reaearch 
Arthur  D.  Little,  Inc. 

Avco  Corp, 

Babcock  and  Wilcox,  F.fcD  Division 
Beech  a/ c  Co. 

Boeing  Airplane  Co. 

Bridgeport  Braes  Co. 

Brunswick  Corp 
Carborundum  Co. 

Clary  Corp. 

Coate  and  Clark,  Inc. 

Dayton,  Ohio 
EC  and  G,  Inc, 

Eeeo  Reaearch 
Fairchild  Hiller 
General  Dynamics 
General  Electric  MSD 
Goerz  Optical  Co. 

Hamilton  Standard 
Headquarters  USAF 
Herculee  Powder  Cc. 

Howmet  Corp. 

Interstate  Steel  Corp. 

Iowa  State  University 
Lawrence  Radiation  Lab. 

Lockheed  California  Co. 

ManLabs. 

Martin 

MIT 


No,  of  Searchi 
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Organisation 


No.  of  Searches 


Nichole  Engineering,  Inc.  1 

Olin  Rerearch  Center  2 

Packard  Hewlett  1 

Pompano  Botch  1 

Racine  and  Vickere,  Armstrong,  Inc.  ! 

Raytheon  1 

Stadia  Corp.  1 

Sieger  Co.  1 

Strata  Industries  1 

Tinker  APB  2 

Turco  Products  1 

University  of  Dayton  21 

U.  S.  Naval  Ordinance  2 

Warren- Law  son  1 

Xanthippe  Chemicale  1 

Unclassified  2 

Total  number  of  eearchee  for  thla  period:  286 


APPENDIX  V 


SUBJECT  CATEGORIES:  DOCUMENT  INPUT  AND  SEARCHES  PROCESSED 

BY  SUBJECT  CATEGORY 


AMIC  COS  All 

01  01 


02  03+04 


03  06+07 


04 


09 


CATEGORY 

Aeronautic* 

Aerodynamic  s 
Aeronautic* 

Aircraft 

Aircraft  flight  control  »r.d  instrument¬ 
ation 

Jet  engine* 

Astronomy,  Astrophysics,  Atmospheric 
Science* 

Aatronomy 

Astrophysics 

Atmospheric  physic* 

Meteorology 

Chemistry,  Rlology,  Medical  Sciences 
Biochemistry 
Bioengineering 
Biology 

Chemical  analysi  s 
Chemical  engineering 
Inorganic  chemistry 
Life  support  systems 
Organic  chemistry 
Physical  chemistry 
Radiochemistry 
Toxicology 

Electronics  and  Electrical  Engineering 
Components 

Electronic  &  electrical  engineering 
Telemetry 


67 


AMIC 

05 


06 


07 


08 

09 

10 
1  1 

12 

13 

14 

15 


COS  ATI 

5 .!  A 


1  i  A 


1  IB 


11C 


CATEGORY 

Adhe  Siva* 

Ceramic  cement! 

Organic  raaln  adhesives 
Petting  compound! 

Seal*,  Sealant* 

Ceramic -metal  bond! 
Mechanical  seals 
O -rings 

Ceramlca,  Refractories,  Glaeaea, 
Mineral! 

Borides 

Carbide! 

Carbon,  graphltea 
Mixed  oxides 
Nitride! 

Single  ox4des 

Coatings,  Paints,  Oxide  Films 


11D  Composites  Material*,  Laminates, 

Sandwich  Structures,  Honecomb 

11E  Fiber!,  Textiles,  Cloth 

11F  Metallurgy,  Metallography 

Alloy! 

Metals 

11H  Otis,  Lubricsnt*,  Heat  Transfer  Fluids, 

Greases,  Hydraulic  Fluids 

111  Polymer*,  Ptaitics 

11J  Elastomers 

11K  Cleaning  Compounds,  Surface  Active 

Agents 


68 


AMIC 

COSATI 

!  b 

1  1L 

17 

21 

18 

1  3 

19 


14 


20 


18 


21 


20 


CATEGORY 

Wood  and  Pap**r  Product* 

Fuels,  Propellants,  Propulsion  System*, 
Explosive* 

Mechanical,  Industrial.  Civil  and  Marine 
Engineering 

Civil  engineering 

Contraction  equipment,  materials, 
supplies 

Containers  and  packaging 

Coupling,  fittings,  fastener#,  joints 

Industrial  processes 

Machining,  tools,  machine  elements 
such  as  bearings,  gas  lubricstlon 
systems 

Marine  engineering 

Pumps,  filters,  pipes,  iittings, 
tubing,  and  valveB 

Safety  engineering 

Structural  engineering 

Methods  and  Equipment 

Apparatus 

Detectors 

Laboratories,  test  facilities,  and 
test  equipment 

Recording  devices 

Nuclear  Science  and  Technology 

F uel  elements;  fuel,  nuclear 

Nuclear  explosions 

Nuclear  power  plants 

Nuclear  reactors 

Radiation  shielding 

Radioactive  wastes 

Physics 

Acoustics 


69 


AMIC 


COSATT 


21  (Continued)  £0 


22  10,  16,  22 


Phyeic  a 

Crystallography 
Electricity  aud  magnetism 
Fluid  mechanic* 

Maters  and  laser* 

Optic* 

Particle  accelerators 
Particle  physics 
Plasma  physic* 

Quantum  theory 
Solid  mechanics 
Solid-state  physic* 
Spectrometry,  spetroscopy 
Thermodynamics 
Wove  propagation 
Space  Technology  and  Missiles 
Astronautics 

Energy  conversion,  solar  cell* 
Launch  vehicles 
Mla tile  technology 
Re -entry  vehicle* 

Rocket* 

Satellite*,  artificial 
Spacecraft 

Trajectories  *nd  re-entry 
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TA But  6, 


WCUMENT  INPUT  AND  SEARCHES  PROCESSED  BY  SUBJECT 
CATEGORY 


AMIC  CATEGORY  DOCUMENT  CATEGORIES  SEARCH  CATEGORIES 

No.  %  No.  % 


01 

139 

2.9 

6 

l.  a 

02 

38 

0.  8 

1 

0.3 

03 

361 

7.  5 

17 

5.0 

04 

137 

2,  9 

18 

5.  3 

05 

20 

0.4 

8 

2.  4 

06 

35 

0.  7 

0 

0.0 

07 

170 

3.5 

20 

5.  9 

08 

116 

2.4 

12 

3.6 

09 

134 

2.  8 

29 

8.6 

10 

30 

0.  6 

4 

1.2 

11 

973 

20.  3 

97 

28.  7 

12 

352 

7.  3 

6 

1.8 

13 

200 

4.  2 

22 

6.  5 

14 

43 

0.9 

4 

1.2 

15 

2 

0.0 

2 

0.  6 

16 

2 

0.0 

e 

0.  0 

17 

54 

1.  1 

l 

0.  3 

18 

363 

7.  6 

31 

9.  2 

19 

404 

8.  4 

14 

4.  1 

20 

173 

3.6 

1 

0.  3 

21 

902 

18.  8 

32 

9.  5 

22 

144 

3.0 

13 

3.8 

71 


APPENDIX  VI 


DESCRIPTION  OP  CAS  SDI  PROGRAM  AND  TYPICAL  USER  FROFIULS 


'a 

The  SDI  ExparUnent  covers  32  journals  ox  general  chemical  interest*’; 
POST-J  covers  article*  from  approximately  425  Journal*  and  Gevernmuni 
report*  specifically  dealing  with  polymer*,  and  POST- P  covers  polymer  re¬ 
lated  patent*  from  20  coustrie*.  POST-P  and  POST-J  are  *ub*cx-ibed  to  for 
member*  of  the  Polymer  Branch  of  the  Nonmei alii <r  Material*  Divis.ou  (MAN} 
of  the  AFML. 

Briefly,  the  operation  of  the  SDI  program  involve*  text  searching  of 
the  title*  and  abstract*  In  conjunction  with  us«r  profiles  by  an  IBM  360  Model 
40  computer,  i'n#  "tilt*"  or  retrieval*  are  determined  by  those  title- abstract 
unit*  which  qualify  according  to  the  user  profile  search  logic.  The  hits  are 
recorded  by  abstract  number  and  bibliographical  data  for  each  user,  and  a 
printing  of  all  abstracts  processed  in  the  particular  period  is  provided.  The 
user  can  then  pick  out  the  abstracts  from  the  printing  which  correspond  to  the 
hits  found  for  hi*  particular  profile. 

The  search  program  has  a  truncating  feature  by  which  portions  of  tech¬ 
nical  words  or  phrases  can  be  employed  in  the  user  profile.  The  advantage  of 
this  truncating  feature  le  that  the  root  of  a  technical  word  can  be  listed  and  the 
text  searching  will  then  retrieve  nil  variations  of  the  root  word  as  long  as  the 
root  itself  appears  in  the  title  or  abstract.  To  Illustrate,  a  scientist  may  be 
interested  in  all  silane  compound*.  By  listing  *SIL>AN*  a*  a  search  unit, 
document  abstracts  on  silane,  dlmethylsiiane,  silanol,  palyellanes,  etc. , 
would  be  retrieved.  If  he  were  interested  la  all  silicon  compounds,  both  or¬ 
ganic  and  lnovganic,  the  root  *SI1>*  might  be  chosen.  There  is  a  possibility 
tli at  noise  or  false  retrieval*  may  occur,  e.  g. ,  the  words  silent,  silk,  tensile, 
and  stiver,  could  cause  false  retrieval  in  the  preceding  example,  but  the  prob¬ 
ability  of  false  retrieval  is  very  low  with  judicious  selection  of  the  truncated 
units.  One  can  specify  that  the  truncation  must  be  the  start  of  a  word,  l.  e. , 
preceded  by  one  or  more  blanks  by  omitting  the  leading  asterisk;  likewise  the 
truncation  can  be  designated  as  the  end  of  a  word  by  leaving  out  the  terminal 
asterisk,  l.e,,  the  truncation  must  be  followed  by  one  or  more  blanks. 

The  user  search  profiles  are  formulated  by  combining  terms  (including 
truncated  terms)  with  AND,  OR,  NOT  logic,  Tn  addition  to  the  AND,  OR,  NOT 
logic,  optional  weighting  factors  can  be  applied  either  in  the  positive  or  nega¬ 
tive  direction.  Thus  undesirable  terms  can  be  given  a  negative  bias  without 
eliminating  them  altogether  a*  would  be  the  case  with  a  strict  NOT  logic  oper¬ 
ator.  With  weighted  terms  on*  specifies  a  minimum  accsptabls  weight  which 
must  be  satisfied  before  retrieval  occurs  (after  all  other  parameters  srs 
satisfied). 
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Thu*  In  the  case  cited  above  of  ,1  roan  desiring  information  on  silicon 
compound*  and  using  *5IL*  a*  a  profile  term,  th*  term*  TENSILE  and 
SILVER  could  be  given  *  negative  weight.  An  abstract  on  tensile  strength 
of  stainless  steel*  would  thus  not  be  retrieved,  but  an  abstract  on  tensile 
elasti  c  constant*  of  single  iryslal  silicon  dioxide  would  satisfy  the  minimum 
weight  rmiuir«m«nii  *nH  thus  be  rf{?Uv£:i  if  tha  weight*  are  carefully 
selected.  ^ 

The  preparation  of  good  user  profiles  is  the  key  to  the  success  of  the 
SDI  program.  User  profile*  were  established  by  a  polymer  chemist  at  A.FML 
in  conjunction  with  the  user#  from  the  Polyene?  Bvaacli.  These  profile*  were 
reviewed  by  a  consultant  from  CAS,  Presently  1*1  profile*  ere  being  search¬ 
ed  on  a  regular  basis.  Two  of  the  profiles  were  modified  considerably  when 
it  was  indicated  by  ueer  feedback  that  too  much  noise  was  occurring.  The 
modifications  of  the  profiles  resulted  in  a  considerable  reduction  in  noia*  but 
very  little,  if  any,  loss  of  pertinent  abstracts  occurred.  Modification  In 
other  profiles  will  undoubtedly  be  made  as  more  experience  is  attained.  Care¬ 
ful  analysis  of  why  the  profile  retrieved  nonpertinent  abstracts  provides  indi¬ 
cation*  as  to  what  modifications  should  be  made  to  improve  performance. 

Typical  examples  of  AFML  search  piofiles  currently  in  use  are  shown 
on  the  following  pages. 
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11  SOOSSOAIMO  M1LIT ARY  ACTIVITY 

AFML  (MAAM) 

Wright- Patterson  AFB,  Ohio  45433 

ii  amtaact  Two  graduate  students  lemistry  were  trained  In  Indexing  of 

technical  documents  through  an  e  a  ished  training  program.  Certain  modifi¬ 
cations  were  made  which  tended  to  oprove  the  training  as  evidenced  by  im¬ 
proved  performance,  A  study  of  irxtra-  and  interindexer  consistency  was  made 
by  subjecting  a  group  of  technical  documents  to  independent  indexing  by  various 
individuals  and  by  two  of  the  same  individuals  one  year  later.  The  concept  of 
essential  index  terms  was  inf  roduced.  Statistical  analysis  showed  that  inira- 
and  interindexer  correlation  tor  experienced  indexers  of  essential  terms  is 
significant  with  a  probability  <p)  >  0.  999.  An  evaluation  of  the  indexer  trainees 
showed  that  essential  terms  was  a  better  performance  criterion  than  total  terms 
and  that  high  interindexer  consistency  was  achieved.  Search  requests  declined 
from  the  preceding  year,  especially  from  the  AFML.  Possible  reasons  for  a 
decline  are  presented.  A  Selective  Dissemination  of  Information  (SDI)  Program 
with  Chemical  Abstracts  was  initiated.  The  development  of  a  vocabulary  and 
thesaurus  for  information  science  related  documents  has  started.  A  free 
indexing  technique  is  being  used. 
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